Cross-validation is a practical and reliable way for testing the predicting power of methods. It's necessary for any machine learning techniques. Even in neural network you need training set, test set as well as validation set to check over optimization.

I also consider it is necessary in neural network training and testing to obtain the mean performance value across n trials.

If you do not have a well separated training and test dataset / or if you are not confident of what percentage of data you should consider for test and training so that there is minimum over fitting or under fitting , cross validation is the best option.

If i use an ensemble of classifiers then is it necessary even......

For ensemble of classifiers I used a double Cross-Validation procedure. Firstly, to generate Learning and Test (test the accuracy of ensemble with patterns not used in learning/training) sets for each trial (repetition of experiments) to get the mean value of performance and standard error. Then, for each trial, I applied again cross-validation to generate the training set (to adapt the weights of the neural network) and the validation set (for early stopping). That provided good accuracy on classification.

As long as you implement strategies that penalise increasing model complexity (regularisation/weight decay) then you know that you are unlikely to overfit the training data, only in this case, you do not need CV, especially when you do not observe a zero error rate.

Usually cross-validation or regularization help. However, on some datasets with thousands of training patterns and not to many inputs, I've observed that the testing error always decreases when you increase the classifier's complexity and number of training iterations.

I hope there is no misunderstanding in terms and I think that by saying "cross validation" you mean the use of a verification subset monitor during NN training for early stopping purposes. Correct? If yes, then it is _absolutely_ necessary. Without it neural networks can be easily overtrained, where you very nicely reproduce the training set observations, but get almost useless model.

If, on the other hand, you understand "cross validation" in terms of post-training testing, like its worst version live-one-out cross validation, then it is not as useful as an external validation.