Every one knows that MLP that used gradient descent suffer from having stuck at local minima and slow convergence rate. Many researhers proposed some modification on parameters such as learning rate, momentum etc. Lately researchers starts to introduce bio-inspired algorithm such ABC, ant colony, cuckoo etc to optimise the wieghts. But not many research done on the other parts which are improving search direction in gradient descent using 2nd order method such conjugate gradient, Newton etc. My question is "are there any other 2nd oder methods that can be use which can produce fast processing?"
The Matlab Neural Network Toolbox provides a dozen or so methods, including several varieties of conjugate gradient and Levenberg-Marquardt. There is no one method that is best for all applications, but you should consult their documentation for comparisons on several benchmark examples. Here is one starting point: http://www.mathworks.com/help/nnet/ug/speed-and-memory-comparison-for-training-multilayer-networks.html
- Badges
- Science topic