How can i reach to a survey or a literature review about the latest researches about the methods of representing the words in language using continuous vectors in the continuous space ?
This from the perspective of the language modeling , as i want to build a statistical language model in the continuous space.
It is like tracking a fox that has a transmitter by
using multiple antennas, where only the amplitudes
can be measured. Every word has a certain semantic
distance to another word in a list of
words, let's call it the basis. These distances
define a vector.
See also the conceptual spaces of Peter Gärdenfors.
Regards,
Joachim
I can suggest some keywords to query:
Natural Language Processing
Vector Space Model
Bag of Words
Entity Resolution
You can find a lot, so you will need to refine your query for the continuos space perspective.
From your post I am gathering that you want to model words and not documents. This presents some challenges when you want to model words in a vector representation. When using cosine similarity in documents the process is to find the position in space between the two documents based on the words within the document. Doing so with words (I guess by using definitions as vectors) will be challenging since the vectors that you will get will be very sparse. Another challenge that you will have is to select the correct sense of the word to compare it against the other word.
If you have more information, then I could possibly suggest other ways around it.
Google's word2vec is an easy-to-use toolkit to represent words as vectors in a continuous space (https://code.google.com/p/word2vec/).
The earlier reference is Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic Regularities in Continuous Space Word Representations. In Proceedings of NAACL HLT, 2013.
Actually this idea came from language modeling, and you can check Mikolov's publications about RNNLM.
In Bag of Words, according to Salton there is not best representation for all collections. I do not remember the reference and I needed it but I read it before and I forget the paper title
- Badges
- Science topic