I have five feature vector(i.e 5 input) and using multi class classification the output is of 3 units (Neurons), So is there any formula to find the number of neurons in hidden layer?, how to find the number of hidden layer?
The right number of hidden layers and the number of neurons within each layer of MLP is hard to find. Although, according to the universal approximation theorem (Cybenko, 1989) for any input-output mapping function in supervised learning, there exists a MLP with NH1 neurons in the hidden layer which is approximately correct. Unfortunately, the theorem does not tell us how to find NH1! Therefore, we generally use a trial and error to find this number NH1. The more neurons in the hidden layer, the better the training error, but the worse the generalization error. The optimal NH1 should be found with cross-validation methods. If the model’s performance ceases to improve sufficiently on the validation set, we can either stop learning (called early-stopping) or modify some optimization (learning rate, add momentum term, regularization, ...).
sqrt(n1,n2) where n1 no. of input neurons and n2 is no of output neurons. This is the most basic formula for finding the no. of neurons in the hidden layers.
There is no perfect rule which gives you the exact number of neurons in hidden layer for the best performance of your neural network. There are many thumb rules available as one of them mentioned by Misha Kakkar, however it may not work for your problem. If you have reviewed previous work on ANN applications in different areas by the researchers wherein they used back propagation neural networks you may get a rough idea what architecture of ANN gives best performance. You may further develop your network based on this information.
obviously the minimum no of hidden nodes is n1 inputs * n2 outputs -there are huge differences in the performance depending on the quality of the "training set" ie the number and reliability of the input-output pairs. Experiment with number of examples and try 'sophisticated' input features and 'smoothing' output classes. See 'back-prop' weight adjustment rules and number of hidden layers as used by experienced users -literature but a lot depends on cplexity of the underlying classification problem.
An easy and fast approach may be using optimization algorithm, such as GA, DE or PSO. My recent paper in Aerospace Science and Technology also deals with similar problems. There are also many researches working on this problem in some journals like Nurocomputing or Neural Networks.
By a carefully planned series of experiments. Rules such as the square route rule are useful in helping you to plan the centre and range of values for your experiment.
The right number of hidden layers and the number of neurons within each layer of MLP is hard to find. Although, according to the universal approximation theorem (Cybenko, 1989) for any input-output mapping function in supervised learning, there exists a MLP with NH1 neurons in the hidden layer which is approximately correct. Unfortunately, the theorem does not tell us how to find NH1! Therefore, we generally use a trial and error to find this number NH1. The more neurons in the hidden layer, the better the training error, but the worse the generalization error. The optimal NH1 should be found with cross-validation methods. If the model’s performance ceases to improve sufficiently on the validation set, we can either stop learning (called early-stopping) or modify some optimization (learning rate, add momentum term, regularization, ...).
There is no hard and fast rule on how many neurons and layers are to be used. But literature suggest two hidden layers are sufficient. Nunber of neurons can be decided by hit and trial method..usually 20-10 neurons are used.
If there is no way to determine the number of neurons in the hidden layer, is it possible to know the lower and upper ranges for this number ? (for example, the total number of weights should be 1/2 the number of training instances ? Lower number of unit is output-1 ?)