Dear all,
I'm finding a solution for extracting the information (as a search engine) from the unstructured data such as the .docx, .xlsx, .pdf, ... files (internal documents).The expected tool is based on Hadoop system. If you have some experiences for this works, please share me some solutions!
Thank You So Much,
Viet
Lucene (http://lucene.apache.org/) is greatly popular search engine for text files and Apache Tika (http://tika.apache.org/) is a powerful text extraction tool.
Building search with Hadoop and Lucene :http://www.mccalv.com/blog/wp-content/uploads/2013/08/apache-hadoop_new_margin-nouni.pdf
Also related links:
http://blogs.sequoiainc.com/blogs/hadoop-lucene-indexing-with-mapreduce
http://blog.cloudera.com/blog/2013/07/the-blur-project-marrying-hadoop-with-lucene/
http://www.cloudera.com/content/cloudera/en/documentation/cloudera-search/v1-latest/Cloudera-Search-Frequently-Asked-Questions/Cloudera-Search-Frequently-Asked-Questions.html
Lucene (http://lucene.apache.org/) is greatly popular search engine for text files and Apache Tika (http://tika.apache.org/) is a powerful text extraction tool.
Building search with Hadoop and Lucene :http://www.mccalv.com/blog/wp-content/uploads/2013/08/apache-hadoop_new_margin-nouni.pdf
Also related links:
http://blogs.sequoiainc.com/blogs/hadoop-lucene-indexing-with-mapreduce
http://blog.cloudera.com/blog/2013/07/the-blur-project-marrying-hadoop-with-lucene/
http://www.cloudera.com/content/cloudera/en/documentation/cloudera-search/v1-latest/Cloudera-Search-Frequently-Asked-Questions/Cloudera-Search-Frequently-Asked-Questions.html
Dear Miss Katarina Grolinger,
It's really useful!
Thank You So Much,
Viet
i'm not sure but i think Apache Solr(http://lucene.apache.org/solr/) might be what you are after.
Hi Mr. Hassan,
As suggested of Miss Katarina, I think Lusence is a good solution. Solr combined Tika and Lucense and it provides a more global solution.
Thank for your answers,
Viet
I would also take a look into using Elasticsearch, which is based on Lucene but has a number of features built over it (like clustering), with Hadoop. Start here: http://www.elasticsearch.org/overview/hadoop/
Hi Mr. Rudi Bonfiglioli,
Thanks for your answer! It's quite interesting! I will consider between these solutions!
Viet
I think you should use Fuzzy logic ,
you can read this paper and if you have any question let me know,
https://www.researchgate.net/publication/273460887_Construction_of_FuzzyFind_Dictionary_using_Golay_Coding_Transformation_for_Searching_Applications
Article Construction of FuzzyFind Dictionary using Golay Coding Tran...
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Analysis
- Data Analysis
- Dataset
- Big Data