I am not an expert, but I think that it has to do with:

1. The number of input elements

2. The number of elements used for training the ANN

3. The number of elements used for test (validate) the ANN

Few hidden units yields a poor aproximation, but numerous hidden units could overtrain the network (the testing error increases).

In MLP the no. of input nodes and output nodes are decided by the problem to be solved. User has to sellect the proper no. of hiddenlayers and no. of nodes in each hidden layer. Training can be started with one hidden layer with 2 nodes. Record an error measure or accuracy measure of the test data set. Next train the network with 3 hidden nodes and record the error/accuracy measure. Repeat this process by increasing the no. of nodes and no. of hidden layers. The size is the size of the NN which optimize the error measure/accuarcy measure.

I think the previous comments are right, and I would like to increase the idea. Another important point is the value of the input. You must normalise those value, but for an application with 10 inputs and 4 outputs, the number of the hidden neurons can be different for another application with the same number of imputs and outputs. Therefore, you have to do a search, and the starting point to find the optimus number of hidden neurons is to apply the following rule

number of hidden neurons=round(sqrt (number of imput*number of output))

On the another hand, the number of layers is another question. Sometimes you can find better results with more layers, but under my experience, it is not much better and the computational time is very very high. It will depend on your application, it is on-line or it can be offline.

Thank you for the suggestions! Do you have by any chance a reference for these handy?

They are nemotechnic rules from working years and meetings with collegues. It is very difficult to have references about practical implementation, but in "Neural Networks for Pattern Recogntion" from Christopher M. Bishop, you can find interesting ideas (Oxford University Press, 1995).

Thanks, I will look them up. I am also aware of using AIC or similar information criteria, but in theory such criteria are inappropriate for neural networks, due to their construction.

To sum up what the others said, the best way perhaps to find the appropriate network structure is to "play and learn", with a rule of thumb start simple and increase the complexity.

As for the references, if you can find papers that have used ANN in applications similar to yours they would be a good source

Basically the size of MLP is determined by the number of input nodes, hidden nodes and output nodes. The number of input nodes can be determined by the number of variables, the number of hidden nodes can be determined by try and error

But basically the rules given below can be a guidance:

Rules for hidden nodes, : h=n, h=2n+1, h= sqrt mn , where m is the number of output nodes, n is the number of input nodes). The number of output nodes usually represent the number of decision we want to make.

But , a little bit different for time series data to determine the input and output, but the simple way to do is : the number of input ==> based on the type of data, if monthly, we use 12, if quarterly we use 4 and etc. the number of output represents the how long we are going to forecast (duration: may be 1-one month, one year, one week or etc...depends on the data we have).

Hope this info can help...

I think that there is no miracle recipe, cross-validation must be performed to determine the optimal parameters while controlling overfitting. In general, it is favored a pyramidal shape with a single hidden layer ( formula may limit the possible range of the number of hidden neurons and √ (Ne + Ns) + a, a: [0,10] (Hui Liu et al 2012).)

I seem to remember that I have read in an article that beyond 2 layers, we did not know what corresponded neither coeffs or the meaning / use of neurons, their connections, etc

to overcome the difficulty of the choice of the architecture of neural networks, it is recommended to combine them with other AI tools such as genetic algorithms that allows to start learning with optimal architecture.Or try to see ANN with RBF instead of the sigmoid function ( The number of hidden neurons is optimized during learning (Gong Li 2010))..

Well, you should only need two layers of x hidden layers ever by Cherbachev's theorem. Of course the number of hidden nodes per layer is difficult.

As Nihad says cross validation is key - you can add a lot of hidden nodes but then you'll start fitting noise. Since each network architecture can come up with good or bad models rerun each layout a few times and cross-validate (personally I've found differences in predictive power to vary from 80% from network models with the same structure.

If the relationships between your inputs and outputs are simple enough, you could use one layer of hidden nodes with as many nodes as you have inputs (that would be like a slightly more complex GLM), if you think there are many interactions, dependencies and such then use a more connected, more complex network.

Also, one last hint would be to dredge out the best GLMM to use as a base line for comparison - you should outperform it easily in cross-validation with new data, if not edit your network.

Hope it helps,

Ben

Benjamin you say that

"f the relationships between your inputs and outputs are simple enough, you could use one layer of hidden nodes with as many nodes as you have inputs (that would be like a slightly more complex GLM), if you think there are many interactions, dependencies and such then use a more connected, more complex network."

please can you give me references in this context.

Thank you

To speed up the cross-validation process, I recommand these optimization tools :

https://github.com/JasperSnoek/spearmint

or

https://github.com/hyperopt/hyperopt

If you use Bayesian regularisation of the neural net, the model becomes relatively insensitive to the number if hidden layer nodes. See burden et al j med chem 1998 or burden et al QSAR comb sci 2009

If you use MLP for classification then you can read my paper about Border pairs method. It is constructive algorithm - much better than backpropagation.

You can also try Bipropagation approach - very reliable and accurate algorithm.

Article Border Pairs Method—constructive MLP learning classification algorithm

The best solution seems to depend from the problem itself and many rules found in bibliography do not always ensure the best topology. Generally, the best geometry is inherent to smaller networks that are capable to capture the relationship between the samples of the training group. However, "trial and error" testings, [Peres, A.M. et al, Chemometr. Intell. Lab., 2011, 105, 65-73 / LeBouf, R.F. et al, Sci. Total Environ., 2010, 408, 3648-3653 / Tagluk, M.E. et al, Expert Syst. Appl., 2010, 37, 1600-1607], and new developed algorithms seem to give very good results [Curteanu, S. and Cartwright, H., J. Chemometr., 2011, 25, 527-549 / Stathakis, D., Int. J. Remote Sens., 2009, 30, 2133-2147].

I agree with the "trial and error" viewpoint, however in case of classification there are a few factors influencing the required number of hidden neurons. If you know (or have assumptions about) your data distribution, it can help. Compact point sets can be bounded around with at least 3 lines in 2D input space (close it in a triangle), 4 planes in 3D input space (tetraeder), and so on: n+1 hyperplanes in nD input space. (Although this separation isn't very smooth, and has sharp vertices.) That is in case of compact classes n+1 hidden neurons per category is enough in a single hidden layer: c*(n+1) (c: number of categories)

Sometimes you don't need to fully bound around your categories, and separation can be opened in some directions, if your class isn't compact in that direction. In this case less hidden layer (than n+1) is sufficient to separate the particular category.

Sometimes the categories' distribution in the input space isn't coherent, one or more class divides itself into one or more (compact) clusters apart from each other. Then you need n+1 hidden neurons to bound around each cluster of a category, and you need an additional hidden layer capable of aggregating distinct clusters into the common category. The first hidden layer: sc*(n+1) neuron (sc: number of subclusters of each class). The second hidden layer: sc neuron (output layer: c neuron)

The real problem is, that you won't know how compact (and separable) your classes really are, and you must try multiple topology.