There are many parameters involved. Here I list the most important ones: Initial weights to start the training, choosing an appropriate learning rate, when we should stop the training, the number of training patterns, splitting dataset into two or three sub-set for training, testing and cross-validation, the type of hidden (and output) unit activation function, the training algorithm, number of neuron in each of hidden layers, ... Too few hidden units will generally leave high training and generalization errors due to underfitting. Too many hidden units will result in low training errors, but will make the training unnecessarily slow, and will result in poor generalization unless some other technique (such as cross-validation, regularization, ...) is used to prevent overfitting.

There are many parameters involved. Here I list the most important ones: Initial weights to start the training, choosing an appropriate learning rate, when we should stop the training, the number of training patterns, splitting dataset into two or three sub-set for training, testing and cross-validation, the type of hidden (and output) unit activation function, the training algorithm, number of neuron in each of hidden layers, ... Too few hidden units will generally leave high training and generalization errors due to underfitting. Too many hidden units will result in low training errors, but will make the training unnecessarily slow, and will result in poor generalization unless some other technique (such as cross-validation, regularization, ...) is used to prevent overfitting.

Thanks for the complete answer Prof. Omid. ANNs are assumed to help us in automation processes, but It seems their tunning and parametrization itself need  automation algorithms !!!  Why do not rely to some non-linear powerful classifiers with much lower parameters to be tunned like SVM ? in SVM just Kernel trick (polynomial or radial or ... ) and its value shall be tunned. just I appreciate any discussion in fore and against

There are numerous classification techniques such as ANN, SVM, ...., but you can not pick one as "good" under all conditions. SVM is ideal small-scale problems and performs well under certain conditions but as you mentioned already the parametrization,  the kernel choice, choice of C and γ are best for a given problem  and the choice of samples have a huge impact on the results.

However, SVMs are not so good for statistical modeling such as regression There is no such limitation with ANNs as they are universal approximator and do well for classification as well as function approximation.

In fact, as far as I understand, there are some reasons ANN dominates the modern classification techniques:

- Automatic training: we can retrain the network anytime we want with new dataset and old trainer.

- Human approach for classification allows the algorithm being used in wider scope of inputs which are usually unpredictable.

However, as Prof. Omid mentioned, choosing a good parameters is the problem of algorithms. If the parameters are not chosen good, the tuning may take very much time to find the best model.

Maybe over 15 parameters are needed to be known. Although maybe some are more important. Over 5 Classifiers are known as appropraite for generalization on nonlinear problems. About 5 algorithms for feature reduction prior to classification are present. About 400 states in front of a user, especially those who just work on applications these techniques in their own branch of science i.e. not speclized in computer science . Maybe knowledge accumulation happens at classifier outputs then we need ensemble mixing (combining) like combination of SVC and MLP y voting, averaging etc.. My massage from this question is not to go for a pre-defined ANN in every problem with a non-scientific push and go strategy. These days it happens : Data --- feed to ANN - Output - Wow nice figures - hasty interpretation of the results.