One the most common approaches to determine the hidden units is to start with a very small network (one hidden unit) and apply the K-fold cross validation ( k over 30 will give very good accuracy) and estimate the average prediction risk. Then you will have to repeat the procedure for increasing growing networks, for example for 1 to 10 hidden units or more if needed.

when you plot the prediction risk vs the number of hidden units (if the training of the network is correct, avoid of local minimums) you will have the following shape: the prediction risk will start to decrease (almost monotonically) and at some point will start to increase (almost) monotonically. The optimal number of neurons is the number that minimizes the prediction risk.

For the input neurons (if I understand correctly, you mean the number of inputs, in other words the number of explanatory variables). If what you mean is to define the correct subset of explanatory variable of a set of possible ones then methods like Sensitivity Based Prunning, average absolute derivate magnitude, PCA or other to name a few can be used. However, I can't say that there is a correct answer in it...it depends on the problem, on the independent variables and on the depended variable

Also note that the Box-Pierce test was designed for linear models not non-linear so the results may not be accurate/true

If you search in the literature, you will get several articles which studied selection of optimal learning rates for ordinary backpropagation algorithm. A few such studies include (i) U. Bhattacharya and S.K. Parui, Self-adaptive learning rates in backpropagation algorithm improve its function approximation performance, Proceedings of the IEEE International Conference on Neural Networks, pp. 2784-2788, 1995, IEEE Computer Society Press, (ii) Kim, H.B., Jung, S.H., Kim, T.G., Park, K.H.: Fast learning method for back-propagation neural network by evolutionary adaptation of learning rates. ;Neurocomputing(1996)101-106. You can try similar methods with your implementation of learning scheme. However, in my experience, if you are interested to get best possible classification accuracy, you should start with small value, say 0.01, and if you are not satisfied with the corresponding classification accuracy, then we should go on increasing the learning rate by fixed but small small steps and each time start training fresh. This may be continued until you will find the improvement in classification accuracy is only marginal. Simply, trial and error method is the most reliable solution for finding the optimal learning rate.

In my experience, the best thing to do is to use a validation set, or, even better to apply cross-validation.

For the first case, you randomly divide your training data in two sets, training and validation. For each number of hidden neurons you want to try, you train your network with the training set and evaluate the error in the validation one. The lowest validation error gives you a good estimation of the best training parameter.

When the number of training points is very low, it is better to apply cross-validation over the traning set, for example, a 10-fold cross-validation and to obtain the cross-validation error for each parameter. This will give you better estimates of the test error.

You might get some helpful ideas in the article mentioned below.

Conference Paper Border Pairs Method – Constructive MLP Learning Classificati...

One the most common approaches to determine the hidden units is to start with a very small network (one hidden unit) and apply the K-fold cross validation ( k over 30 will give very good accuracy) and estimate the average prediction risk. Then you will have to repeat the procedure for increasing growing networks, for example for 1 to 10 hidden units or more if needed.

when you plot the prediction risk vs the number of hidden units (if the training of the network is correct, avoid of local minimums) you will have the following shape: the prediction risk will start to decrease (almost monotonically) and at some point will start to increase (almost) monotonically. The optimal number of neurons is the number that minimizes the prediction risk.

For the input neurons (if I understand correctly, you mean the number of inputs, in other words the number of explanatory variables). If what you mean is to define the correct subset of explanatory variable of a set of possible ones then methods like Sensitivity Based Prunning, average absolute derivate magnitude, PCA or other to name a few can be used. However, I can't say that there is a correct answer in it...it depends on the problem, on the independent variables and on the depended variable

Also note that the Box-Pierce test was designed for linear models not non-linear so the results may not be accurate/true

Number of neurons depend on the your input data set, mostly if you have 500 samples of data 20 to 50 hidden neurons is optimum, it also depend how your code is complex, and in my opinion the better hidden neurons are to start check 20, 25, 30,35 to 50, 55.

Number of input neurons must be equal to number of features/attributes in input vector, hidden neurons can be be at least 1.5 times of input neurons to start with, and can be increased as experimentation proceeds, one can find out the best performing configuration.

It depends on the number of variables, size of data sets, validation methods, accuracy of the obtained results, data types, neural methods, quality of data sets and the nature of your data.

Ieroham Baruch. My opinion is than the number of hidden neurons ought to be equal to the sum of the inputs and outputs.

there is no theory yet to determine how many hidden neurons are needed to approximate any given function. If there are too few hidden neurons, a high training error and high generalization error would result due to under-fitting. On the other hand, if there are too many hidden neurons, there would be a low training error, but there would still be a high generalization error due to over-fitting. In most situations, there is no way to determine the best number of hidden neurons without training several networks and estimating the generalization error of each. To do this there are two methods: network prunning and network growing technique.

I recommend these references:

1.Guang-Bin, H., C. Lei, and S. Chee-Kheong, Universal approximation using incremental constructive feedforward networks with random hidden nodes. Neural Networks, IEEE Transactions on, 2006. 17(4): p. 879-892.

2. Haykin, S.S., Neural networks: a comprehensive foundation1994: Macmillan.

One important guideline is that the number of weights+bias (the total number of parameters) to be found must be less than the number of the training points. Otherwise the NN will be over trained and will have a low generalization capability.

These are no generalized rule to select the number of neurons in the hidden layer. It varies from problem to problem. The best way to select the optimum number of neurons is to do repeated testing by changing the number of neurons.

A thumb rule is that to use a single hidden layer with (number of inputs+number of outputs)/2 neurons... just a thumb rule.

Antonio Dourado has provided a good guideline to avoid overfitting. You can start from [Number of Training Samples/2*(Number of Input Neuron+Number of Output Neurons). It works in most of the cases but if your problem is too simple or complex, you may try increasing or decreasing the number of neurons accordingly.

See you are using a recurrent neural network so it has dynamic nonlinear property due to internal feedback loops. So, it won't require much neurons to work with. Infact use less number of neurons say 10. It will work.

@ Dr. Manasi Patil, do you have a reference paper for your statement?