There are several ways. A few, in case of feedforward and feedback topologies can be like as follows:

1. Have multiple layers with higher number of neurons than usually allowed by Kolmogorov limit.

2. Have multiple number of hidden layers with heterogeneous activation functions. 

3. Construct multiple ANNs with the first feeding inputs to the next and sustaining the training. 

Some relevant reading are as below:

http://neuralnetworksanddeeplearning.com/chap4.html

http://radar.oreilly.com/2014/07/how-to-build-and-run-your-first-deep-learning-network.html

https://timdettmers.wordpress.com/2015/03/26/convolution-deep-learning/#more-192

http://spectrum.ieee.org/tech-talk/computing/software/biggest-neural-network-ever-pushes-ai-deep-learning

Did you try it?

https://developer.nvidia.com/deep-learning

here you can find most deep learning algorithms and their implementation

http://deeplearning.net/tutorial/

http://deeplearning.net/software/theano/

http://keras.io/

http://torch.ch/

Torch will be the best option if you have huge data like ImageNet

Hi Chen,

well the question is how much yourself. Try H20 for deep learning and neural network. One advise the documentation is not in sync with the latest version of the package.

Have a look:

http://h2o.ai/product/deep-learning/

and check the package h2o in R. 

Alain

I would strongly advice you take a good look at Python. In the moment I guess it's the most popular language for data science and machine learning. There is a great community and you can find specific modules for almost any application. A very important aspect especially for Deep Learning is the ability to use parallel computation. In Python it is very easy to for example with Theano to take advantage of the GPU and perform data-intensive calculations up to 140x faster than with CPU [1]. The authors of Theano provide a tutorial which makes it very easy to get startet and build a handwritten digits recognizer in very short time. I think this is the best place to start.

If you want to go further and build more complex networks you should look at Apache Spark. It easily integrates with Python and is super scalable. There are two courses I would recommend looking at for spark, Introduction to Big Data with Apache Spark [2] and Scalable Machine Learning [3], both from Berkeley.

I haven't worked with Torch yet, but it is true, that it is used at least at Facebook. I think Yann LeCun is one of the creators and that might be the reason why. And he is one of godfathers of Deep Learning. But I can't really say much about that. Maybe you get something out of this comparison to Theano [4]. 

Oh, I just remembered, one of the most interesting publications in my field, Reinforcement Learning, provides their code for a Deep Reinforcement Learning approach using Torch [5].

[1] http://deeplearning.net/software/theano/

[2]https://courses.edx.org/courses/BerkeleyX/CS100.1x/1T2015/info

[3] https://courses.edx.org/courses/BerkeleyX/CS190.1x/1T2015/info

[4] http://fastml.com/torch-vs-theano/

[5] http://www.nature.com/nature/journal/v518/n7540/full/nature14236.html

Very informative links:

• http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/
• https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/
• https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/

There are many tools to implement ANN and DNN.

For beginners, Matlab is a nice way. For serious and goal oriented works, tools like Caffe and Torch are useful. Python is a popular approach. Nowadays people are preferring TensorFlow. It facilitates use of GPUs as well which cuts down programming time and accelerates training.

NVIDIA has a range of solutions to accelerate DNN learning. Some of these are

https://developer.nvidia.com/deep-learning

https://developer.nvidia.com/deep-learning-software

Please read following paper:

And Please look at these Github repository:

1- https://github.com/kk7nc/RMDL

2- https://github.com/kk7nc/HDLTex

3- https://github.com/BarnesLab/Patient2Vec

Please let me know if you have any questions

also you can see:

https://github.com/kk7nc/RMDL

A Complete Guide on Getting Started with Deep Learning in Python

FAIZAN SHAIKH, AUGUST 31, 2016 LOGIN TO BOOKMARK THIS ARTICLE

Introduction

Deep Learning, a prominent topic in Artificial Intelligence domain, has been in the spotlight for quite some time now. It is especially known for its breakthroughs in fields like Computer Vision and Game playing (Alpha GO), surpassing human ability. Since the last survey, there has been a drastic increase in the trends. (click here to check out the survey)

Here is what Google trends shows us:

📷

If you are interested in the topic here’s an excellent non-technical introduction. If you are interested to know the recent trends, here’s a great compilation.

Here our aim is to provide a learning path to all those who are new to deep learning and also the ones who want to explore it further. So are you ready to step onto the journey of conquering Deep Learning? Let’s GO!

Step 0 : Pre-requisites

It is recommended that before jumping on to Deep Learning, you should know the basics of Machine Learning. The Learning Path on Machine Learning is a complete resource to get you started in the field.

If you want a shorter version, here it is:

• Basics of Math (Resource 1: “Math | Khan academy” (Especially Calculus, Probability and Linear Algebra))
• Basics of Python (Resource: “Intro to Computer Science”, edX course)
• Basics of Statistics (Resource: “Intro to Stats”, Udacity course)
• Basics of Machine Learning (Resource: “Intro to Machine Learning”, Udacity course)

Timeline : Suggested: 2-6 months

Step 1 : Setup your Machine

Before going on to the next step, make sure you have the supported hardware. It is generally recommended that you should have atleast

• A good enough GPU (4+ GB), preferably Nvidia
• An OK CPU (eg. Intel Core i3 is ok, Intel Pentium may not be)
• 4 GB RAM or depending upon the dataset.

If you are still unsure, go through this hardware guide.

PS: If you are a hardcore gamer (not just candy crushers obviously!), you may already have the required hardware.

If you don’t have the required specifications, you could either buy it or lease an Amazon Web Service instance. Here’s a good guide for using AWS for deep learning.

Note: Do not install any deep learning libraries at this stage, do it on step 3.

Step 2 : A Shallow Dive

Now that you have a good enough knowledge of pre-requisites, you should go on further into understanding Deep Learning.

As per your preference you could follow:

• Blog Approach : (Resource 1: “Fundamentals of Deep Learning”, Resource 2: “Hacker’s guide to Neural Networks”)
• Video Approach : “Deep Learning Simplified”
• TextBook Approach : (“Neural networks and Deep Learning”)

Along with the pre-requisites, you should get to know the popular deep learning libraries and the languages for running them. Here’s a (non-comprehensive) list (Check the wiki page for a more comprehensive list):

• Caffe
• DeepLearning4j
• Tensorflow
• Theano
• Torch

Some other notable libraries include Mocha, neon, H2O, MXNet, Keras, Lasagne, Nolearn. Here’s a list of Deep Learning libraries by Language.

Check out Lecture 12 of Stanford’s CS231n course for a brief overview of some of the popular libraries.

Timeline : Suggested 1-3 weeks

Step 3 : Choose your own Adventure!

Now comes the interesting part! Deep Learning has been applied in various fields with state-of-the-art results. To get a taste of this side of the moon, you, the reader, gets to choose which path to take. This should be a hands-on experience, so that you get a proper foundation on what you have understood until now.

Note: Each path contains a primer blog, a practical project, the required deep learning library for the project and an assisting course. First go through the primer, then install the required libraries and get on with the project. If you face any difficulties along the way, use the associated course to back you up.

• Deep Learning for Computer Vision
  • Primer : “DL for Computer Vision” blog.
  • Project : “Facial Keypoint Detection” Tutorial
  • Required libraries : Nolearn
  • Associated Course : “CS231n: Convolutional Neural Networks for Visual Recognition”
• Deep Learning for Natural Language Processing
  • Primer : “Deep Learning, NLP, and Representations” blog.
  • Project : “Deep Learning for Chatbots”: “Part 1”, “Part 2”
  • Required library : Tensorflow
  • Associated Course : “CS224d: Deep Learning for Natural Language Processing”
• Deep Learning for Speech/Audio
  • Primer : “Deep Speech: Lessons from Deep Learning” news article and corresponding video.
  • Project : “Music Generation using Magenta (Tensorflow)”
  • Required library : Magenta
  • Associated Course : “Deep Learning (Spring 2016), CILVR Lab@NYU”
• Deep Learning for Reinforcement Learning
  • Primer and Project : “Deep Reinforcement Learning: Pong from Pixels”
  • Required Library : No deep learning library required. Although you do require openAI gym to test your model.
  • Associated Course : “CS294: Deep Reinforcement Learning”

Timeline : Suggested 1-2 months

Step 4 : Deep Dive into Deep Learning

Now you are (almost) ready to make a dent in Deep Learning Hall of Fame! The path ahead is long and deep (pun intended) and mostly unexplored. Now it is upto you to make use of this newly acquired skill as efficiently as you can. Here are some tips you should do to hone your skill.

• Reiterate the above step with a different adventure.
• Deep Learning for none of the above! (eg. DL for trading, DL for optimizing energy efficiency)
• Use your newly learned skills to build something (Remember, with great power; comes great responsibility)
• Test your Deep Learning skills (eg. kaggle)
• Participate in Deep Learning community. (eg. Google Group, DL Subreddit)
• Follow recent researches / researchers. (eg. “RE.WORK DL Summit”)

Timeline : Suggested – Infinity!

Noteworthy Resources

• Complete Deep Learning book
• Stanford UFLDL Tutorial
• “Deep Learning in Neural Networks: An Overview”, a survey paper on Deep Learning
• Awesome Deep Learning github repository
• Yann LeCun’s recommendations for Deep Learning self-study

End Notes

I hope this learning path was helpful to you. I have tried to make it as comprehensive as possible. Now, it’s time for you to practice and read as much as you can. To gain expertise in working in neural network try out our deep learning practice problem – Identify the Digits.

Once you have an understanding of Deep Learning and its associated concepts, take the Deep Learning Skill test. The way Deep learning is gaining recognition it is important to be familiar with it.

Good luck! Did you like reading this article? Do you follow a different approach / package / library to get started with Deep Learning? I’d love to interact with you in comments.

Hey Chen Ziheng

I recommend you to please go through the below link for Deep Learning-

https://data-flair.training/blogs/deep-learning-with-python-tutorial/