Hello everyone,
Recently, I have read some articles about deep learning but I don't understand what is exactly different between traditional ANN and Deep Learning? Since, it seems that most of concepts are same in both issues! I want to know the pros and cons on these methods in comparison with each other?
I want to say thank you in advance for your kind support.
Dear Yousefnezhad,
Before deep learning was invented, almost all neural networks researchers cling themselves to the new SVM bandwagon. Two reasons for this: (1) Adding more layers to MLP does not works (e.g., to compete with SVM) due to "diminishing error problem" (i.e., the error from the output layer that was back propagated to the inner layer getting small and small, hence in short the MLP infact does not learn"; (2) Three layers MLP was mathematically explained to be a unversal approximator -- in short there was no reason to add more layers;
Deep learning addresses the first problem (reason) by inventing pre-training (greedy layer wisely learn before move up to the higher layers) and fine-tuning (correct the unsupervisedly learn weights using few labeled data). Deep learning addresses the second reason by offering more abstraction complexity and hierarchical features learning through many of its layers.
The big different between ordinary machine learning technique and deep learning is that the ordinary machine learning usually use hand-crafted features, whereas in deep learning the features are unsupervisedly crafted by machine through its deep structure. Such new hierarchical features lead to the currently best recognition performance of deep learning in many cases.
The main shortcomings, and infact also the advantage, of deep learning is the requirement of big data to unsupervisedly crafting the features.
As with neural networks, I found that we are still lacking a comprehensive theoretical basis to validate the results of deep learning performances. Without sound theoretical basis, it seems the field currently felt more as magic instead of science. I believe we can hope to gain more lights in this aspect.
I summarize and review some of these deep learning algorithms. I hope this would help.
http://fananymi.tumblr.com/archive
Dear Yousefnezhad,
Before deep learning was invented, almost all neural networks researchers cling themselves to the new SVM bandwagon. Two reasons for this: (1) Adding more layers to MLP does not works (e.g., to compete with SVM) due to "diminishing error problem" (i.e., the error from the output layer that was back propagated to the inner layer getting small and small, hence in short the MLP infact does not learn"; (2) Three layers MLP was mathematically explained to be a unversal approximator -- in short there was no reason to add more layers;
Deep learning addresses the first problem (reason) by inventing pre-training (greedy layer wisely learn before move up to the higher layers) and fine-tuning (correct the unsupervisedly learn weights using few labeled data). Deep learning addresses the second reason by offering more abstraction complexity and hierarchical features learning through many of its layers.
The big different between ordinary machine learning technique and deep learning is that the ordinary machine learning usually use hand-crafted features, whereas in deep learning the features are unsupervisedly crafted by machine through its deep structure. Such new hierarchical features lead to the currently best recognition performance of deep learning in many cases.
The main shortcomings, and infact also the advantage, of deep learning is the requirement of big data to unsupervisedly crafting the features.
As with neural networks, I found that we are still lacking a comprehensive theoretical basis to validate the results of deep learning performances. Without sound theoretical basis, it seems the field currently felt more as magic instead of science. I believe we can hope to gain more lights in this aspect.
I summarize and review some of these deep learning algorithms. I hope this would help.
http://fananymi.tumblr.com/archive
- Badges
- Science topic