A newly developed Information Retrieval system requires its testing against the existing solutions. Besides evaluation metrics, often comprehensive datasets are required so that the system can be tested and evaluated. It is therefore, requested to share your ideas and suggestions on developing datasets that can make evaluation process easier, accurate, and precise.
Thanks in advance
Sir Irfan,
In order to build a dataset, its required to collect data about that domain for which you are going to design dataset. for Example if somebody want to develop a dataset for movies recommendation systems then he should crawl data about movies and users from concerned sources such as IMDB. These information may include movie director, actor, actress, producer, genre, ratings and user id and other users related information. These information can be store in CSV format and could be used for assessing proposed movie recommender systems.
Regards
Zafar Ali
Irfan,
Take a look at :
A Large Benchmark Dataset for Web Document Clustering by Mark P. Sinka and David W. Corne
Some of the challenges in deriving a dataset for information retrieval is the dependence of the initial selection of documents based on search engines. The best thing to do is to extract samples from as many search engines as possible to avoid bias.
Regards
Irfan, we are going through this process at the moment for new and complex software.
I'd suggest you start with 'obvious' sources of data such as out of copyright books or almanacs of relevant data 'where anyone can read the book and understand it' ; then comparing one system to another is easier to the assessor as we now have a three way test not a two way, where both systems could agree yet both still be wrong!
After that build up on the deviousness as you gain confidence wit the new systems as you know the effort and resources will payoff.
I have a colleague in Lahore who might guide you on this. Can connect you up if that helps? Let me know.
regards
Ken
Thank you Kenneth for your valuable response. I am looking forward for further help and assistance.
Best Regards
Irfan
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Human-Computer Interaction