As it is described and proposed in some references such as [1], 5x2 cross-validation is a suitable approach for comparing two algorithms and we use it in our work. But, if we want to optimize the hyper-parameters of an algorithm for example MLP learning, (decide its controllable factors, e.g. the number of hidden layers or hidden neurons) are we forced to use the same method of 5x2 cross-validation that we use for comparing our algorithm with others. Or we can use 30-replications of the train-test method?

[1] Dietterich, T. G. 1998. Approximate Statistical Tests for Comparing Supervised

Classification Learning Algorithm, Neural Computation 10: 1895–1923.

Similar questions and discussions