Hi Huma,

First, I recommend you to follow this machine learning tutorial, in case you do not have knowledge in this field.

http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

It is very important to understand the concepts of neural networks, backpropagation, and so on, to go deeper in the deep learning classifiers, since deep learning can be seen (in a coarse view) as neural networks with a larger number of hidden layers. Let's say that classical machine learning neural networks have 1-3 hidden layers, while deep learning approaches use more hidden layers.

Cheers.

Hi Huma,

First, I recommend you to follow this machine learning tutorial, in case you do not have knowledge in this field.

http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

It is very important to understand the concepts of neural networks, backpropagation, and so on, to go deeper in the deep learning classifiers, since deep learning can be seen (in a coarse view) as neural networks with a larger number of hidden layers. Let's say that classical machine learning neural networks have 1-3 hidden layers, while deep learning approaches use more hidden layers.

Cheers.

I'd recommend a standard text book on machine learning first, if you are new to classifiers. For deep learning, take a look at e.g. Geoffrey Hinton's work.

This book also might be of help: http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf

Thank you everyone. I've rolled up the sleeves and jumped into machine learning.

https://www.coursera.org/course/ml

I recommend you this course, Machine learning in Coursera, taught by Andrew Ng in Stanford

This lecture is easy to follow and also you can learn how to implement machine learning algorithm in matlab because programming assignment is based on the matlab code.

I am also into Deep learning now, and i find this new area of neural network was really fascinating. You could search up cat paper by Google team, that uses millions of data for training with 16,000 machines. 

Dear Chaudhry,

In my opinion the easiest way to understand deep learning is to start from neural networks that I belief most of us know this well.

Before deep learning was invented around 2006, almost all neural networks researchers cling themselves to the new SVM bandwagon. Two reasons for this: (1) Adding more layers to MLP does not works (e.g., to compete with SVM) due to "diminishing error problem" (i.e., the error from the output layer that was back propagated to the inner layer getting small and small, hence in short the MLP infact does not learn"; (2) Three layers MLP was mathematically explained to be a unversal approximator -- in short there was no reason to add more layers;

Deep learning addresses the first problem (reason) by inventing pre-training (greedy layer wisely learn before move up to the higher layers) and fine-tuning (correct the unsupervisedly learn weights using few labeled data. Deep learning addresses the second reason by offering more abstraction complexity and hierarchical features learning through many of its layers.

The big different between ordinary machine learning technique and deep learning is that the ordinary machine learning usually use hand-crafted features, whereas in deep learning the features are unsupervisedly crafted by machine through its deep structure. Such new hierarchical features lead to the currently best recognition performance of deep learning in many cases. 

The main shortcomings, and infact also the advantage, of deep learning is the requirement of big data to unsupervisedly crafting the features.

As with neural networks, I found that we are still lacking a comprehensive theoretical basis to validate the results of deep learning performances. Without sound theoretical basis, it seems the field currently felt more as magic instead of science. I believe we can hope to gain more lights in this aspect.

I summarize and review some of these deep learning algorithms. I hope this would help.

https://fananymi.wordpress.com/

http://fananymi.tumblr.com/archive

Dear Chaudhry and All, 

I would like to revise my answer on the origin and inventorship of deep learning (DL). Based on the following link:

http://people.idsia.ch/~juergen/deep-learning-conspiracy.html

Actually, DL was started much earlier, by Alexey Grigorevich Ivakhnenko in 1971. Praphrasing Juergen Schmidhuber in the following paper:

http://arxiv.org/abs/1404.7828

"A paper of 1971 already described a deep GMDH

network with 8 layers (Ivakhnenko, 1971). There

have been numerous applications of GMDH-style

nets, e.g. (Ikeda et al., 1976; Farlow, 1984; Madala

and Ivakhnenko, 1994; Ivakhnenko, 1995; Kondo,

1998; Kord´ık et al., 2003; Witczak et al., 2006;

Kondo and Ueno, 2008)."

Somehow, LeCun, Bengio, and Hinton, overlooked or missed those earlier work to cite in their papers but heavily cite each other works.

Maybe later on, after studying their open responds to the refutation, I may revise my answer again. 

Sincerely, 

 My suggession is to follow this course from  Standford. It is very nice and applied. They do not bore you with all mathematical derivations, rather focusing on understanding. Here is the link: 

http://cs231n.stanford.edu/syllabus.html

I was wondering where I can find a matlab sample code for regression based counting using Deep Learning?