Things Deep Learning is (currently) not so good at include
a) Data-efficient learning: Sometimes you simply don't have enough data (or you can't collect it fast enough), especially when you work with physical/mechanical/biological systems (e.g., robots, biological systems). Here, experimentation is costly (time, money) and you would require a method that extracts useful information from a relatively small data set. If you do this, then you also should be
b) Providing uncertainty about your model/prediction, another point that requires more attention in deep learning, especially when you hook up your model to a decision-making system (i.e., a system that uses the outcome of the model to assess the expected risk/utility). If you have plenty of data, this uncertainty is very small, and you may ignore it. However, if your data set is relatively small (or you want to learn fast/data-efficient), you should express this uncertainty.
c) Regression. Most Deep Learning algorithms seem to focus on classification or dimensionality reduction. Regression is not so much in the focus of Deep Learning. A reason for this may be that the "old" neural networks have problems with regression, too (overfitting).
Things Deep Learning is (currently) not so good at include
a) Data-efficient learning: Sometimes you simply don't have enough data (or you can't collect it fast enough), especially when you work with physical/mechanical/biological systems (e.g., robots, biological systems). Here, experimentation is costly (time, money) and you would require a method that extracts useful information from a relatively small data set. If you do this, then you also should be
b) Providing uncertainty about your model/prediction, another point that requires more attention in deep learning, especially when you hook up your model to a decision-making system (i.e., a system that uses the outcome of the model to assess the expected risk/utility). If you have plenty of data, this uncertainty is very small, and you may ignore it. However, if your data set is relatively small (or you want to learn fast/data-efficient), you should express this uncertainty.
c) Regression. Most Deep Learning algorithms seem to focus on classification or dimensionality reduction. Regression is not so much in the focus of Deep Learning. A reason for this may be that the "old" neural networks have problems with regression, too (overfitting).
Thank you Marc. A good point noted here is data efficient learning. I believe that data plays a deterministic role in the performance of any AI system. As such, it is and important facet to understand the importance of coding in Deep learning.
Yes, data is very important in AI. However, there are impressive success stories in AI (with deep learning) where you don't work with physical systems. For instance, a few weeks ago, Google DeepMind publishes a paper in Nature (see link below) where they applied deep learning to autonomously learn to play computer games (at human level), purely based on pixel information from the computer screen. Since you can simulate millions of games (it's just a matter of time, but not so much money or hardware) you can feed the learning system a lot of data.
http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html
Dear Mohabeer,
I have just wrote a summary and review on the most intriguing paper on deep learning. There is indeed blindspots inside decision space of deep learning:
http://fananymi.tumblr.com/post/113944071239/review-on-the-most-intriguing-paper-on-deep
Another link which looks into the paper is might be also helpful.
http://www.i-programmer.info/news/105-artificial-intelligence/8064-the-deep-flaw-in-all-neural-networks.html
Biological neuroevolution and deep learning (a machine learning technique) are two different things. Many scientists still feel that everything regarding building a reliable thinking machine can be reduced to machine learning.
For a real progress in AI we will need far more http://dx.doi.org/10.13140/2.1.2286.5608
Dear Mohabeer,
I would like to add a clarification on Adversarial Examples
http://www.kdnuggets.com/2015/01/deep-learning-flaws-universal-machine-learning.html
thanks to the following research done by Ian Goodfellow:
http://www.kdnuggets.com/2015/07/deep-learning-adversarial-examples-misconceptions.html
I hope this would help.
- Badges
- Science topic