I'm quite interested in Natural Language Processing and familiar with general structure of Neural Networks and so I wish to find my starting point in NLP field. To do that, I want to know what are the current best models for different NLP tasks. I've heard about GPT models but I want to know about other NLP models too. Which are other ones you can recommend to me?
BERT and GPT uses a Transformer architecture that only requires fine-tuning on the specific natural language task. BERT improves GPT approach by using a pre-training task that is more likely to capture bidirectional context.
Other Transformer-based language models include Google’s ALBERT, CMU’s XLNet, Facebook’s RoBERTa, HuggingFace’s DistilBERT, Saleforce’s CTRL, Nvidia’s Megatron, ByteDance’s AMBERT, and Stanford’s ColBERT.
Efficient Language Models:
- Small Feed-Forward Networks,
- The Evolved Transformer,
- PRADO + pQRNN,
- MobileBERT,
- SqueezeBERT,
- Lite Transformer with Long-Short Range Attention,
- MicroNet,
- DeLighT
I think Google's BERT. OpenAI is also popular.
Preprint BERT: Pre-training of Deep Bidirectional Transformers for La...
GPT is one of the most famous NLP models because it is good at many things rather than being excellent in just one. GPT is a transformer model and transformer models are better than old-school RNN models. Other examples of famous transformers are BERT and ELMo.
BERT and GPT uses a Transformer architecture that only requires fine-tuning on the specific natural language task. BERT improves GPT approach by using a pre-training task that is more likely to capture bidirectional context.
Other Transformer-based language models include Google’s ALBERT, CMU’s XLNet, Facebook’s RoBERTa, HuggingFace’s DistilBERT, Saleforce’s CTRL, Nvidia’s Megatron, ByteDance’s AMBERT, and Stanford’s ColBERT.
Efficient Language Models:
- Small Feed-Forward Networks,
- The Evolved Transformer,
- PRADO + pQRNN,
- MobileBERT,
- SqueezeBERT,
- Lite Transformer with Long-Short Range Attention,
- MicroNet,
- DeLighT
- Badges
- Science topic