What could be the approaches to combine the pairwise document similarity scores to get the overall similarity score of a certain document against a document collection?
You might, for example, calculate the centroid of the document collection and then calculate the similarity between the document and the centroid; you might calculate the average from similarities between the document and all documents from the collection; you might calculate the similarity as the minimum/maximum (i.e., the most similar and the least similar) from all similarities between the document and all documents from the collection… As the similarity measure, you might use, e.g., cosine similarity or Euclidean distance.
Topic Modeling techniques like Dirichlet smoothing, TF-IDF, the Latent Dirichlet Allocation (LDA) and Segmented Topic Model (STM) can be used.
You can use a Bag-Of-Words approach to extract keywords and then group based on them. I used a similar approach for grouping news articles. If you need more info, you can find it here
http://www.ijcjournal.org/index.php/InternationalJournalOfComputer/article/view/685/398
There are well-documented specific software tools for this. One of them is Copycatch
http://eprints.sztaki.hu/3093/
You can also use KNIME https://www.knime.org if you are use to build ETLs
You might, for example, calculate the centroid of the document collection and then calculate the similarity between the document and the centroid; you might calculate the average from similarities between the document and all documents from the collection; you might calculate the similarity as the minimum/maximum (i.e., the most similar and the least similar) from all similarities between the document and all documents from the collection… As the similarity measure, you might use, e.g., cosine similarity or Euclidean distance.
- Badges
- Science topic