We are working on the hyperparameter optimization of neural networks so for more exposure latest optimization techniques needed for experiment. Anyone kindly suggest me latest optimization algorithms.
There are many techniques proposed for hyper-parameter optimisation in neural networks. In practice, Grid Search, Random Search and Bayesian Search are widely used for hyper-parameter optimisation.
All these optimisation are used to select the best hyper-parameter based on the given set of values to each hyper-parameter but in evolutionary based optimisation of hyper-parameter can evolve new set of hyper-parameters with different combinations.
Many approaches were proposed over the years, the most popular approach is NEAT and many variants like Hyper-NEAT and HA-NEAT also introduced for hyper-parameter optimisation. This application of evolutionary algorithms on neural networks to evolve various topologies, weights, activation and other hyper-parameters can be known as Neuroevolution.
- Badges
- Science topic