There are several ways. A few, in case of feedforward and feedback topologies can be like as follows:

1. Have multiple layers with higher number of neurons than usually allowed by Kolmogorov limit.

2. Have multiple number of hidden layers with heterogeneous activation functions. 

3. Construct multiple ANNs with the first feeding inputs to the next and sustaining the training. 

Some relevant reading are as below:

http://neuralnetworksanddeeplearning.com/chap4.html

http://radar.oreilly.com/2014/07/how-to-build-and-run-your-first-deep-learning-network.html

https://timdettmers.wordpress.com/2015/03/26/convolution-deep-learning/#more-192

http://spectrum.ieee.org/tech-talk/computing/software/biggest-neural-network-ever-pushes-ai-deep-learning

There are several ways. A few, in case of feedforward and feedback topologies can be like as follows:

1. Have multiple layers with higher number of neurons than usually allowed by Kolmogorov limit.

2. Have multiple number of hidden layers with heterogeneous activation functions. 

3. Construct multiple ANNs with the first feeding inputs to the next and sustaining the training. 

Some relevant reading are as below:

http://neuralnetworksanddeeplearning.com/chap4.html

http://radar.oreilly.com/2014/07/how-to-build-and-run-your-first-deep-learning-network.html

https://timdettmers.wordpress.com/2015/03/26/convolution-deep-learning/#more-192

http://spectrum.ieee.org/tech-talk/computing/software/biggest-neural-network-ever-pushes-ai-deep-learning

Did you try it?

https://developer.nvidia.com/deep-learning

here you can find most deep learning algorithms and their implementation

http://deeplearning.net/tutorial/

http://deeplearning.net/software/theano/

http://keras.io/

http://torch.ch/

Torch will be the best option if you have huge data like ImageNet

Hi Chen,

well the question is how much yourself. Try H20 for deep learning and neural network. One advise the documentation is not in sync with the latest version of the package.

Have a look:

http://h2o.ai/product/deep-learning/

and check the package h2o in R. 

Alain

I would strongly advice you take a good look at Python. In the moment I guess it's the most popular language for data science and machine learning. There is a great community and you can find specific modules for almost any application. A very important aspect especially for Deep Learning is the ability to use parallel computation. In Python it is very easy to for example with Theano to take advantage of the GPU and perform data-intensive calculations up to 140x faster than with CPU [1]. The authors of Theano provide a tutorial which makes it very easy to get startet and build a handwritten digits recognizer in very short time. I think this is the best place to start.

If you want to go further and build more complex networks you should look at Apache Spark. It easily integrates with Python and is super scalable. There are two courses I would recommend looking at for spark, Introduction to Big Data with Apache Spark [2] and Scalable Machine Learning [3], both from Berkeley.

I haven't worked with Torch yet, but it is true, that it is used at least at Facebook. I think Yann LeCun is one of the creators and that might be the reason why. And he is one of godfathers of Deep Learning. But I can't really say much about that. Maybe you get something out of this comparison to Theano [4]. 

Oh, I just remembered, one of the most interesting publications in my field, Reinforcement Learning, provides their code for a Deep Reinforcement Learning approach using Torch [5].

[1] http://deeplearning.net/software/theano/

[2]https://courses.edx.org/courses/BerkeleyX/CS100.1x/1T2015/info

[3] https://courses.edx.org/courses/BerkeleyX/CS190.1x/1T2015/info

[4] http://fastml.com/torch-vs-theano/

[5] http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html

https://www.slideshare.net/mohamedloey/deep-learning-71352250

Very informative links:

• http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/
• https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/
• https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/

There are many tools to implement ANN and DNN.

For beginners, Matlab is a nice way. For serious and goal oriented works, tools like Caffe and Torch are useful. Python is a popular approach. Nowadays people are preferring TensorFlow. It facilitates use of GPUs as well which cuts down programming time and accelerates training.

NVIDIA has a range of solutions to accelerate DNN learning. Some of these are

https://developer.nvidia.com/deep-learning

https://developer.nvidia.com/deep-learning-software

Please read following paper:

And Please look at these Github repository:

1- https://github.com/kk7nc/RMDL

2- https://github.com/kk7nc/HDLTex

3- https://github.com/BarnesLab/Patient2Vec

Please let me know if you have any questions

also you can see:

https://github.com/kk7nc/RMDL