Conventional Gradient Descent is very slow for Deep Learning training. While investigating about alternative methods to train Deep Neural Networks faster, I came across a few algorithms like Stochastic Gradient Descent, Contrastive Divergence, Optimization Heuristics etc. I am looking for the resources to explore all such important methods to fasten up Deep Learning training and parameter optimization time.
I'd appreciate any lead about such resources and some clarity about Contrastive Divergence algorithm? Is it an approximation to Gradient Descent or is it a different algorithm altogether? Thanks.
Dear Puneet,
I suggest you to see links, attached files and publications in topics.
-Restricted Boltzmann Machines (RBM) - Deep Learning
deeplearning.net/tutorial/rbm.html
-A Practical Guide to Training Restricted Boltzmann Machines
https://www.cs.toronto.edu/~hinton/absps/guideTR.pdf
-VLSI Implementation of Deep Neural Network Using Integral ... - arXiv
https://arxiv.org/pdf/1509.08972
-Deep Learning for Multivariate Financial Time Series - KTH
https://www.math.kth.se/matstat/seminarier/reports/M.../150612a.p...
-A Tutorial on Stochastic Approximation Algorithms for Training ...
citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.178...
Best regard
Article Stopping Criteria in Contrastive Divergence: Alternatives to...
Article Machine Learning Approximation Algorithms for High-Dimension...
Article Learning RBM with a DC programming Approach
Article A Hybrid Machine Learning Ensemble Approach Based on a Radia...
Dear Mohamed-Mourad Lafifi sir,
The papers - 'A Fast Contrastive Divergence for RBM' and 'Stopping Criteria in Contrastive Divergence' added to my understanding of Contrastive Divergence. And, the other links you shared helped me get some idea about the various stochastic approximation attempts to fasten up Deep NN training. Thanks !
Dear Mr. Ebadi, really helpful papers and articles, thanks for sharing.
- Badges
- Science topic