The number of hidden neurons is very syndicate since the hidden neurons are regarded as the processing neurons in the network, plus having a small number of hidden neurons can increase the speed of the training session whereas a large number of hidden layers can prolong the training session. There are two techniques for selecting this parameter, these are namely; growing technique via which the number of hidden neurons is selected as a small number then the number gets increased gradually. The other technique is called pruning via which the number of hidden neurons is selected as a large number and then it gets decreased via eliminating some insignificant components .  It is recommended to select the initial number of hidden in accordance with the equation one , then after, either the Growing method or Pruning is used to arrive at the best selection of hidden neurons so that to achieve promising results for the network.

hidden no= (input no. +output no.)/2                      eqaution one.

context no= no of hidden

Hi

So far, no suitable method proposed to determine the optimal number of neurons in ANN and all calculations is done on the basis of trial and error. In the implementation work, the number of neurons is selected as the lowest of the hardware used and in the optimization and computational problems, the number of neurons is chosen such that the minimum error is achieved. To select the number of neurons in ANN you should trade off between hardware use (such as FPGA slices), error and calculation time.

Good Luck

I'd recommend starting with the tiling algorithm of Mézard and Nadal, http://lptms.u-psud.fr/membres/mezard/Pdf/89_MN_JPA.pdf and going on from there.

no suitable method is giving idea about optimum hidden layer. it depend on the training set and data patron. 2 hidden layer training is faster and error is also less.

Sure there is, cf. the tiling algorithm. It's astonishing that theoretical knowledge from over 20 years ago isn't common knowledge for the applications, where it's relevant.

One large sized layer with appropriate regularization scheme should be enough as neural network is a universal function approximator.

No, it's well known that there are associations that a one-layer network can't represent, namely those that aren't linearly separable; this was noted by Minsky and Papert in 1969; the modern theory was developed by E. Gardner in 1988-89 and the solution for any rule was found by Mézard and Nadal, in the paper linked to above, where the relevant literature can, also, be found.

Hi,

To select the optimal value of hiperparameters, such as number of hidden neuron, usually we learn and test NN with different number of neurons in the cross-validation procedure, and we choose the best one.

By the way usually one hidden layer with nonlinear neurons is enough to approximate any continuous function (universal approximation property).

Best regards,

You probably can try NEAT, a neuroevolutionary algorithm, which can evolve ANN with minimal hidden neurons.

The statement that neural networks in  general and one-layer networks in particular have the ``universal approximation property'' is a mathematical statement that can be subject to proof-and can be proved to be wrong. The proof for the one-layer networks was given by Minsky and Papert and the general framework was developed by Gardner. Cf. Mézard & Nadal. It can be proven that such networks CANNOT approximate certain rules, i.e. associations. On the other hand it can be proven that  it is possible to construct a network that CAN represent certain associations and to prove how well it can do so.

Let me explain a bit about NEAT (as mentioned by Min Shi) and why that could be a solution for such problems. I have been applying AR and various types of ANNs (e.g. WNNs) to predict time series. Optimizing the structure of the ANNs is manual and it's still the responsibility of the algorithm designer to find it. An issue which exists on optimizing the parameters (meta-problem); however NEAT aims to provide a problem independent solution by augmenting the architecture of the neural network and adjusting the weights. NEAT might not be as precise as a manual designed ANN for a specific problem but it's reusable and mobile. If you need high precision and you're not switching between different problems our suggestion might not be suitable for you. But you want to go toward the automated problem independent solution then you may want to consider NEAT. 

You need to use trial method for obtaining best results.

Indeed, while the tiling algorithm constructs *a* network that can represent the association-or rule-a very interesting question is how to incorporate additional constraints and how these may be interpreted as constraints and not as part of the rule s such. 

At the moment, there is not an specific algorithm to determine the optimal number of hidden layers. You must use a method of trial and error. Usually, two hidden layers are used when you have a set of patterns very uncorrelated or in the case of time series when you have seasonal changes or strong changes in the tendency. In any case, there is not a established rule. 

No, that's not correct. Please study the theory-that exists for 20 years. You do not *need¨ to use a method of trial and error-that's a personal choice, not imposed by the state of the art. It's possible to understand and describe quantitatively the consequences. Please don't confuse statements of personal opinion with statements of mathematics. 

Please, could you supply references? I'm very interested in these algorithms and their applications to environmental time series.

I did-cf. my first message on this thread, the references therein and the citations.The thread is only 2 pages long...

Cf., also, http://lptms.u-psud.fr/membres/Mezard/Pdf/89_M_JPA.pdf 

The number of hidden neurons is very syndicate since the hidden neurons are regarded as the processing neurons in the network, plus having a small number of hidden neurons can increase the speed of the training session whereas a large number of hidden layers can prolong the training session. There are two techniques for selecting this parameter, these are namely; growing technique via which the number of hidden neurons is selected as a small number then the number gets increased gradually. The other technique is called pruning via which the number of hidden neurons is selected as a large number and then it gets decreased via eliminating some insignificant components .  It is recommended to select the initial number of hidden in accordance with the equation one , then after, either the Growing method or Pruning is used to arrive at the best selection of hidden neurons so that to achieve promising results for the network.

hidden no= (input no. +output no.)/2                      eqaution one.

context no= no of hidden

Dear Gerro:

You may read my article about applying neural networks to predict the thermodynamic properties of refrigerants. In short: I solved the problem of finding the best number of neurons by incrementing the hidden layer number of neurons (form 5 to 11), and comparing their errors (MSE). I also applied a non-parametric statistical test in order to compare the errors of different ANN models, so I could be sure that the selected model is really the best. If you need my article, I will be glad to share with you. 

Kind regards.

http://www.sciencedirect.com/science/article/pii/S0140700714001856

refer to the paper:

CURVE FITTING FOR COARSE DATA USING ARTIFICIAL NEURAL NETWORK, WSEAS TRANSACTIONS on MATHEMATICS, C. Balasubramanyam, M. S. Ajay, K. R. Spandana, Amogh B. Shetty, K. N. Seetharamu

http://www.wseas.org/multimedia/journals/mathematics/2014/a045706-398.pdf

Can you tell me How to decide the type of sigmoid to be used in different layers

It shouldn't, really,  matter, since it can only affect the rate of convergence, assuming a solution exists. The quantities that determine the sigmoid and that are sensitive to any scale, will, obviously, be sensitive to any particular scale the association has. 

I've read that for continuous output one layer is sufficient. For discrete output two layers are recommended. I've searched for the best solution (one or two layers) in my thesis. As two layers haven't shown better prediction accuracy, I've focused on finding optimum number of neurons in one hidden layer.

Have you found the right answer?

Best regards

Hubert

I faced the same problem when I needed to generate an artificial neural network for predicting thermodynamic properties. I solved by using a kind of automated "trial and error" process. I started with one hidden layer and 5 neurons. Later I move on to 6, 7,...15 neurons keeping one hidden layer. Therefore I generated 11 ANN models. Then I performed an error analysis by comparing the Mean Square Error (MSE) of each model and the model with the lowest MSE was selected as the best predicting model. You can do the same process and repeat for 2 hidden layers, and use different transfer functions (logsig, tansig...). If you need more precise selection criteria, you could use a Wilcoxon test. I described the above process in my article. (you can find it on the Research Gate page or send me an email).

Kind regards.

I have found one web article on neural net that implemented with R programming, illustrating a for loop for selecting optimum number of hidden neurons. You can try this with any kind of deep learning techniques.

Website link:

https://www.r-bloggers.com/selecting-the-number-of-neurons-in-the-hidden-layer-of-a-neural-network/

Most of the people go for a 10 fold cross validation test. Doing it manually is one of the biggest pain. So make it automated with for loop.

I hope you will find this information helpful.