http://colinraffel.com/wiki/neural_network_hyperparameters

Dear Chethankumar,

reading your question, I thought the following website might be appreciated by you. It is a summary on hyperparameters in deep learning.

https://deeplearning.web.unc.edu/files/2016/02/Hyperparameter-Optimization-for-Deep-Learning.pdf

Yours sincerely,

Rainer

Hi, you might also find these two links useful:

https://www.coursera.org/learn/neural-networks-deep-learning/lecture/TBvb5/parameters-vs-hyperparameters

https://towardsdatascience.com/what-are-hyperparameters-and-how-to-tune-the-hyperparameters-in-a-deep-neural-network-d0604917584a

Dear Prayag , Rainer and Meareg

Thank you for your suggestions ..I will go through the contents

Dear Sir,

I would say the hyper-parameters are most important things in the model. For example, number of hidden layers and number of unites in hidden layer, and regularization parameters. These parameters will decide the model is over fitting or under fitting on a specific data set.

An extreme example would be: if we set hidden layer to be 1 and 1 hidden unit, with regularization parameter 10 million. The model will provide nothing.

Dear Sir;

What I believe is that the central challenge in training deep architectures is to deal with the strong dependencies that exist during training between parameters between layers (or skills levels). To address this difficulty of the problem we must simultaneously have two questions:

- adapt to low layers to provide adequate information for the final adjustment (end of training) of the above layers,

- adapt the layers above to make good use of the final adjustment (end of training) of the lower layers.

But this will not be easy, the parameters still have to be chosen, yes it is only one way.

Hi,

This link also may help:

https://blog.floydhub.com/guide-to-hyperparameters-search-for-deep-learning-models/

The question is unclear to me. You want to know the importance of hyperparameters or role of each hyperparameter. More or less all of the parameters are important but they also depend on the loss function or objective function you use. For instance, in image denoising, choosing the right receptive field is very important but while dealing with action recognition you can just start with any and trivially change it with respect to the layers you design. Optimizer such as Adam works better in image denoising and single image super resolution but in most action recognition cases SGD or Ada delta optimizer dominate. There are alot of other parameters with respect to the network you design and the optimization for each of them is necessary in order to achieve the optimal results. I would recommend you to read Deep Learning book by Ian GoodFellow. The parameters are explained in detail let it me the use of dropout ratio or early stopping it covers all and in easily readable manner

Hyperpameters are an essential part of any deep network that helps you to optimize the quality of the network. Change in parameters, helps you to get the desired results or soulution to your problem.

It is Optimization for quality of network. Kind of deep architecture That can have strong dependency between layers

See https://www.researchgate.net/publication/324876042_Multi-threaded_learning_control_mechanism_for_neural_networks

As typical deep learning libraries and frameworks instantiates deep learning algorithms with default parameters. Therefore, it is always recommend to do hyper-parameters search to find the optimum set of combinations. This is important as fitting an algorithm on different datasets requires different parameters values in order to fully converge and find the absolute (global) minimum.