I think the biggest obstacle to Deep Learning is the need to have HUGE amount of data.  For big universal problems like image recognition, that's no problem but for specific problems that small or medium size business might encounter, that's often different...

Deep Network requires a huge computing power to train.

To answer your question, I suggest you these readings of interest :

• https://drive.google.com/file/d/0BxKBnD5y2M8NVHRiVXBnOVpiYUk/edit associated with http://www.cs.tau.ac.il/~wolf/deeplearningmeeting/pdfs/lecun-20141105-tau-intel-master-class.pdf
• http://www.stat.berkeley.edu/~bruna/Media/meetup_may24.pdf
• https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2016-23.pdf
• https://www.google.fr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjApIemrPLTAhWIMhoKHYBpAUoQFggrMAA&url=http%3A%2F%2Fceur-ws.org%2FVol-1802%2Fpaper5.pdf&usg=AFQjCNESsI5rC96XwPhzAV28cA_fTyLdHg

As a global overview addressing the deep learning paradigm with future perspectives, here is a valuable reference: https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf

One of the main challenges nowadays commonly identified is to add reasoning capabilities to learning ones.

Regards

The principal limitations of DL are: very time-consuming (it is computationally very expensive, and this can be solved by applying parallelization), difficult to configure (we could define a methodology for a proper parameter configuration based on many experiment results) and not interpretable results (the complexity of the hidden layers of DL makes it difficult to interpret the results or to understand the algorithm mechanism).

Cite: Miralles-Pechuán, L., Rosso, D., Jiménez, F., & García, J. M. (2016). A methodology based on Deep Learning for advert value calculation in CPM, CPC and CPA networks. Soft Computing, 1-15.

With recent advances and better understanding in deep learning, large amounts of data and learning duration are becoming less and less limiting factors.

A recent reference ( https://arxiv.org/abs/1509.09308 ) proposes an efficient solution using Winograd's minimal filtering ( https://www.nervanasys.com/winograd/ ) to face with learning data sets of small/medium sizes in addition to big data already efficiently processed by means of conventional FFT ( https://arxiv.org/abs/1312.5851 , https://arxiv.org/abs/1511.01870 ).

Moreover, deep learning powered by (possibly hierarchical) spectral clustering opens new interesting perspectives in the future in terms of knowledge modeling, information encoding/representation and interpretation ( https://arxiv.org/abs/1312.6203 , https://arxiv.org/abs/1611.00336 , http://www.cl.uni-heidelberg.de/courses/ws14/deepl/BengioETAL12.pdf ).

For interested readers, here is a very recent paper by Gu et al. on "Recent Advances in Convolutional Neural Networks" (and deep learning), 2017: https://arxiv.org/pdf/1512.07108.pdf

This paper extends my previous post in relation with Winograd and FFT.

Lake et al. wrote a very recent paper (2016) about the future of deep learning entitled "Building Machines That Learn and Think Like People ": https://arxiv.org/pdf/1604.00289.pdf

Here is an excerpt from this paper:

"...Since the birth of AI in the 1950s, people have wanted to build machines that learn and think like people. We hope researchers in AI, machine learning, and cognitive science will accept our challenge problems as a testbed for progress. Rather than just building systems that recognize handwritten characters and play Frostbite or Go as the end result of an asymptotic process, we suggest that deep learning and other computational paradigms should aim to tackle these tasks using as little training data as people need, and also to evaluate models on a range of human-like generalizations beyond the one task the model was trained on. We hope that the ingredients outlined in this article will prove useful for working towards this goal: seeing objects and agents rather than features, building causal models and not just recognizing patterns, recombining representations without needing to retrain, and learning-to-learn rather than starting from scratch..."

A proposition of interest in order to initiate some reasoning capabilities in artificial neural networks:

Johnson et al.  suggest "a model for visual reasoning that consists of a program generator that constructs an explicit representation of the reasoning process to be performed, and an execution engine that executes the resulting program to produce an answer. Both the program generator and the execution engine are implemented by neural networks, and are trained using a combination of Backpropagation and Reinforcement". Follow https://arxiv.org/pdf/1705.03633.pdf

Neurosciences, valuable source of inspiration for machine learning by following https://www.researchgate.net/post/Why_deep_neural_network

The main limitation and challenging task in Deep learning is to reduce the complicity of network. In normal deep learning technique there are more than 20 layers. 

Researcher have been working to reduce the number of layers and get good accuracy.

There are several limitations.

Have a look into this one: https://arxiv.org/pdf/1711.03985.pdf (specifically Section 4: Open Issues and Future Perspectives)

Good luck!

Although the field of deep learning produces good results. However there are few limitations

1. It depends upon the amount of data. If the large quality of data is fed to the input layers of neurons it in a general sense perform well in many cases.

2. The underlying process of what is happening at the neuronal level is not clear. Speaking clearly the functionality is not understood.

3. It is computationally expensive.

4. The algorithms are not able to learn through noisy labels.

Dear All Colleagues ;

It is a good question and helpful answers.

The training process using deep learning techniques, has, a high computational cost and it is a time consuming process, therefore, it is better to use parallel processing tools, such as graphics processing units (GPUs), to accelerate the training process of large datasets.

Thanks and regards