Dear All,
I found the question when I tried to answer the following discussion:
https://www.researchgate.net/post/Is_there_an_effective_method_to_control_the_depth_of_a_deep_learning_network
Is there any theoretical lower bound on the number of layers in deep learning?
A multillayer perceptron with 2 layers, implies a hidden layer-and that can solve, certain classes of, nonlinear problems. I don't understand why people insist on making statements, that are mathematical in content, in a metaphysical way. The theory is known for more than 20 years-one example is the tiling algorithm, linked to above and is based on the work of E. Gardner and of B. Derrida and E. Gardner, cf. http://www.lps.ens.fr/~derrida/PAPIERS/1988/optimal-storage.pdf for the general construction and the paper of Mézard and Nadal and by Kocher and Monasson for the general multilayer case.
Yes: cf. Mézard and Nadal, http://lptms.u-psud.fr/membres/mezard/Pdf/89_MN_JPA.pdf
and Kocher and Monasson, http://www.phys.ens.fr/~monasson/Articles/a02.pdf
Dear Mohamad,
No, there is neither minimum nor maximum. It is generally assumed that deep learning refers to learning weights in a network with more than 3 layers. For some problems, a 3 layer neural network might perform just as well as a deep network with more layers. However, a multi layer perceptron (MLP) with only two layers cannot solve nonlinear problems.
A multillayer perceptron with 2 layers, implies a hidden layer-and that can solve, certain classes of, nonlinear problems. I don't understand why people insist on making statements, that are mathematical in content, in a metaphysical way. The theory is known for more than 20 years-one example is the tiling algorithm, linked to above and is based on the work of E. Gardner and of B. Derrida and E. Gardner, cf. http://www.lps.ens.fr/~derrida/PAPIERS/1988/optimal-storage.pdf for the general construction and the paper of Mézard and Nadal and by Kocher and Monasson for the general multilayer case.
Dear Stam and Andrea,
Thank you very much for your answer. Let me study them in depth while waiting for other answer.
- Badges
- Science topic