I wonder if word embeddings such as fasttext or word2vec really improve classification results. Are there any studies that acknowledge significant improvement of classification of natural/language texts (>5%).
Dear Alexander Osherenko,
1- https://towardsdatascience.com/nlp-performance-of-different-word-embeddings-on-text-classification-de648c6262b
2- chrome-extension://oemmndcbldboiebfnladdacbdfmadadm/https://arxiv.org/pdf/1901.09785.pdf
3- Preprint Exploring Word Embedding Techniques to Improve Sentiment Ana...
Have a look at https://medium.com/@tatheerhussain_28261/a-quick-overview-of-the-main-difference-between-word2vec-and-fasttext-b9d3f6e274e9
Article A survey of word embeddings for clinical text
What is still not clear to me why they calculate results with word embeddings e.g. CBOW although maybe only BOW is sufficient? Why don't they use BOW as a base line?
The BoW based techniques ignores the context.
In BoW you get the words and their count regardless the position of these words in the text, although the position mattees.
For that reason and to overcome this limitation , word
G
Yes, I am aware of the context. But still, why should I consider the overhead of word embeddings if they don't improve the BOW results. To consider the context, it is possible to use grammar rules.
G
I recently worked on a research project to classify the "risk type" in construction documents. The paper is under review and when publish I can share that.
Basically, I Compared the following NLP models:
1. TF-IDF + (support vector machine (SVM) with Stochastic Gradient Descent (SGD), Logistic Regression (LR), and Bernoulli Naïve Bayes (BNB))
2. Word2vec + (SVM, LR, BNB)
3. FasTtext + (SVM, LR, BNB)
4. Bidirectional Encoder Representations from Transformers (BERT)
BERT model outperforms other NLP models significantly.
If interested check out:
https://towardsdatascience.com/multi-class-text-classification-with-deep-learning-using-bert-b59ca2f5c613
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification