Dear Chinedu Nwoye ,

The term "deep learning" was coined in 2006, and refers to machine learning algorithms that have multiple non-linear layers and can learn feature hierarchies [1].

Most modern machine learning relies on feature engineering or some level of domain knowledge to obtain good results. In deep learning systems, this is not the case -- instead, algorithms can automatically learn feature hierarchies, which represent objects in increasing levels of abstraction. Although the basic ingredients of many deep learning algorithms have been around for many years, they are currently increasing in popularity for many reasons, including advances in compute power, the falling cost of computing hardware, and advances in machine learning research.

https://www.kaggle.com/c/word2vec-nlp-tutorial/details/what-is-deep-learning

Regards, Shafagat

No, Word2Vec is not a deep learning model, it can use continuous bag-of-words or continuous skip-gram as distributed representations, but in any case, the number of parameters, layers and non-linearlities will be too small to be considered a deep learning model.

See for example the diagram in:

Rong, Xin. "word2vec parameter learning explained." arXiv preprint arXiv:1411.2738 (2014).

Where you can see that just one encoder-decoder matrix W is learned.

Also, you can check Christopher Manning's lecture on Word2Vec here:

https://www.youtube.com/watch?v=ERibwqs9p38&list=PL3FW7Lu3i5Jsnh1rnUwq_TcylNr7EkRe6&index=2

Cheers,

Sebastian

"Depth of architecture refers to the number of levels of composition of non-linear operations in the function learned. Whereas most current learning algorithms cor

respond to shallow architectures(1, 2 or 3 levels), the mammal brain is organized in a deep architecture with a given input percept represented at multiple levels of abstraction" [1]

I could also argue based on my own research that an additional factor is based on  the initial research carried out in NN's where using two hidden layers was computationally expensive (especially using second order methods). This led the scientific community to focus on single hidden layer architectures and naming multiple hidden layers as deep. While Hiinton's discovery has propelled the deep architecture paradigm, new research on GPU's have also facilitated the deployment of deep architectures.

[1] Learning Deep Architectures for AI by Yoshua Bengio

word2vec itself is a simple bi-layered neural network architecture, it turns text into meaningful vectors form that deeper networks can understand. In other words the out put of simple neural word2vec model is used as input for Deep Networks.