What is the best way right now to measure the text similarity between two documents based on the word2vec word embeddings?
We used word2vec to create word embeddings (vector representations for words).
Now we want to use these word embeddings to measure the text similarity between two documents.
Which technique it the best right now to calculate text similarity using word embeddings?
Thanks.
Here is an interesting work was done by Matt Kusner :
http://mkusner.github.io/publications/WMD.pdf
If you want to use the cosine distance, averaging word vectors (trained using word2vec) for document embedding may help you:
https://lirias.kuleuven.be/bitstream/123456789/500032/3/SIGIR2015TalkVulicMoensSlides.pdf
You can use Cosine similarity as a metric to calculate text similarity.
http://blog.christianperone.com/2013/09/machine-learning-cosine-similarity-for-vector-space-models-part-iii/
.
you may find the following link interesting :
http://emnlp2014.org/tutorials/8_notes.pdf
.
Here is an interesting work was done by Matt Kusner :
http://mkusner.github.io/publications/WMD.pdf
If you want to use the cosine distance, averaging word vectors (trained using word2vec) for document embedding may help you:
https://lirias.kuleuven.be/bitstream/123456789/500032/3/SIGIR2015TalkVulicMoensSlides.pdf
Paragraph vectors I think perform pretty well. link -https://research.google.com/pubs/pub44894.html
They derive their roots from word2vec.
High level implementation of this paper is provided by gensim -
https://radimrehurek.com/gensim/models/doc2vec.html
These are some ways to do the same (in decreasing order of simplicity):
1. Using vector space model and computing cosine similarity. (Doesn't use embeddings)
2. Using tf-idf with cosine similarity. (Doesn't use embeddings)
3. Averaging word embeddings over the sentence to get document vector and calculating cosine similarity. (Uses word embeddings)
4. Supervised Learning approach: Training a similarity function between documents. The state of the art method right now is the LSTM based dual encoder model. (Uses word embeddings)
If it is semantic approach then I would recommend Latent Semantic Indexing.
Evaluating the Impact of Word Embeddings on Similarity Scoring
https://dl.gi.de/bitstream/handle/20.500.12116/3987/B29-2.pdf?sequence=1
Doc2Vec is a similar method to word2vec and is better for document similarity.
Euclidian similarity and Cosine Similarity can be used. But the Euclidian similarity may not work well for the high-dimensional word vectors since the Euclidian similarity may increase as the number of dimensions increases. The cosine similarity measures and captures the angle of the word vectors and not the magnitude, the total similarity of 1 is at a 0-degree angle while no similarity is expressed as a 90-degree angle.
In the following links usage cases can be found :
Thesis Applying Machine Learning and Natural Language Processing Te...
Conference Paper BUSEM at SemEval-2017 Task 4A Sentiment Analysis with Word E...
Conference Paper Turkish tweet sentiment analysis with word embedding and mac...
- Badges
- Science topic