Hi,
maybe someone knows where I can find a webpage dataset to Information extraction evaluation. I need a set like a:
- domain_1 = { {web_page_1, {relevant entities}}, ..., { {web_page_2, {relevant entities} }
I created a wrapper induction algorithm with based on domain's web pages. This algorithm can extract an important entity from these pages (for example from domain about movies they from each page extract information like film title, actors names etc.) . I created a reference dataset (I labeled 3 domain and 200 documents). But maybe there is an another better reference dataset?
Maybe someone know where I can find a software to comparation with my solution (semi-supervised information extraction from web pages based on html structure) ?
Please have a look on http://tinyurl.com/o8ykn4y. This is the dataset used to evaluate a recent work on IE at web scale. The full description of this work and the way the corpus was extracted is described in http://www.aclweb.org/anthology/D15-1086. More information within the project's page of this group of authors http://oak.dcs.shef.ac.uk/lodie/
I hope this link helps..https://archive.ics.uci.edu/ml/datasets.html
here you can find various datasets.
You can try ClueWeb12 dataset from The Lemur Project at http://www.lemurproject.org/clueweb12.php. It consists of more than 700.000.000 web pages. Although it's not free, it helped me a lot in my past researches.
Hi thanks for a response but, unfortunately this resources can't help me in my research. I just find another dataset:
- https://wwwdb.inf.tu-dresden.de/misc/dwtc/
- http://www.iesl.cs.umass.edu/data
- http://zitnik.si/mediawiki/index.php?title=Datasets#Datasets
- http://mogadala.com/Toolkits_and_Datasets.html
- http://sherlock.ics.uci.edu/data.html
- http://www.lemurproject.org/clueweb09.php/
- http://www.cs.technion.ac.il/~gabr/resources/data/ne_datasets.html
- http://www-nlp.stanford.edu/software/web-entity-extractor-ACL2014/paper/www/index.html
But they also haven't a required labeled html dataset. I need a dataset - html documents like a:
doc_1 = "...
Hope these links could help you.
Good luck.
http://spraakbanken.gu.se/metashare/repository/search/
https://catalog.ldc.upenn.edu/
http://trec.nist.gov/data.html
I found two well-labelled data sets:
1. http://www.dia.uniroma3.it/db/weir
Bronzi, M., Crescenzi, V., Merialdo, P., Papotti, P.: Extraction and integration of partially overlapping web sources. PVLDB 6(10), p. 805-816 (2013)
2. http://swde.codeplex.com/
Hao, Q., Cai, R., Pang, Y., Zhang, L.: From one tree to a forest: a uni ed solution for structured web data extraction. In: Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. p. 775-784. ACM (2011)
DMOZ on the link https://dmoz-odp.org. It is the largest webpages archive.
- Badges
- Science topic
- Similar topics
- Political Science
- International History and Politics
- Globalization