When you have a very small dataset, I would split it nevertheless in training / validation / test set, even if the vali and test set would contain only one record (leave-one-out). But then I would repeat the tuning experiment many times with different vali and test sets to gather enough statistical validity. If you use only one test set, your results may be biased towards the specific situation of this test. set.

Whether you use CV or random subsampling is in my personal opinion a matter of choice. I've experimented a lot in recent years with the focus on parameter tuning  for data mining and the questions of proper train/vali/test set splitting. You can read more about possible choices in the attached technical report (Konen, Wolfgang; Koch, Patrick (2012): The TDMR Package: Tuned Data Mining in R.  Cologne University of Applied Science,  02/2012 ). The chapter "TDMR Data Reading and Data Split in Training / Validation / Test Data" may be especially helpful. The R-package TDMR is also available as open source from CRAN. Another source of valuable information concerning training-test-set-splitting is Bernd Bischl's package mlr (machine learning in R) and the literature connected to this.

Hope this helps!

http://maanvs03.gm.fh-koeln.de/webpub/CIOPReports.d/Kone12a.d/Kone12a.pdf

Dear friend

Greetings.

Incase the sample size is very small, you can not use cross validation.

You need to use separate sample set for training and testing.

I hope this helps you.

Best regards

Dr.Indrajit Mandal, Ph.D.

I think the only way to create robust machine learning model for your system is increasing your samples by investing more resources to do it. Because every algorithms that merge two steps of parameter optimization and system evaluation phase make your model more overfit.

Hi, log average performances over x-fold (e.g. 10) non-overlapping test sets with one parameter combination, then change parameter values, test, log average over non-overlapping test sets and repeat until at some point you find the global optimum. Attached is a RapidMiner code, which optimizes LibSVM rbf kernel parameters C and gamma using 10-fold CV evaluation.

Thank a lot dear friends for your beneficial comments,

Unfortunately because of the nature of the problem that we are faced with, we can't increase the sample size. So I decided to consider CV or Leave one out error as a estimation for test error. But I'm looking for piece of advice from different researchers.

Thanks a lot

In your case, maybe you have to resort to some expert or prior information.

When you have a very small dataset, I would split it nevertheless in training / validation / test set, even if the vali and test set would contain only one record (leave-one-out). But then I would repeat the tuning experiment many times with different vali and test sets to gather enough statistical validity. If you use only one test set, your results may be biased towards the specific situation of this test. set.

Whether you use CV or random subsampling is in my personal opinion a matter of choice. I've experimented a lot in recent years with the focus on parameter tuning  for data mining and the questions of proper train/vali/test set splitting. You can read more about possible choices in the attached technical report (Konen, Wolfgang; Koch, Patrick (2012): The TDMR Package: Tuned Data Mining in R.  Cologne University of Applied Science,  02/2012 ). The chapter "TDMR Data Reading and Data Split in Training / Validation / Test Data" may be especially helpful. The R-package TDMR is also available as open source from CRAN. Another source of valuable information concerning training-test-set-splitting is Bernd Bischl's package mlr (machine learning in R) and the literature connected to this.

Hope this helps!

http://maanvs03.gm.fh-koeln.de/webpub/CIOPReports.d/Kone12a.d/Kone12a.pdf

I am not sure it can be applied in the specific case but, maybe, you can use "nested" cross-validation, which is described in this book:

https://books.google.it/books?id=UNY8O7-Og0wC&printsec=frontcover&dq=%22A+Gentle+Introduction+to+Support+Vector+Machines%22&hl=it&sa=X&ei=X3emVMDhMMfuUr2Sg8gL&ved=0CCAQ6AEwAA#v=onepage&q=%22A%20Gentle%20Introduction%20to%20Support%20Vector%20Machines%22&f=false

and in the related slides:

http://webdoc.nyumc.org/nyumc/files/chibi/user-content/Final.pdf

Agreeing with Giorgio above here. Nested resampling is probably exactly what you want, and I would say standard in such a situation. 

Splitting in training and testing (which is called "Holdout") is a strategy you use when you have many data points. Apparently you operate on the other end of the spectrum. In this case we use the statistical trick of "repeating" this process (said very roughly) to reduce variance in estimation, and to kind of ensure maximal usage of data for training and testing. What results are methods like Leave-One-Out, Subsampling, CV, or repeated CV.

Nested resampling now uses that process BOTH for tuning (or in general: model-selection) and selected model evaluation. You have 2 nested (hence the name) loops, one for model optimization in the inside, and one for model eval on the outside. This can become computationally more expensive, so you kind of trade in runtime for a more reliable estimation. 

Although I am trying, it is hard to explain all the details in such a post as here. I encourage you to read up properly on the subject of model evaluation. Getting this right is one of your major responsibilities as a predictive modeler - and many people unfortunately don't. 

Here is a paper I often recommend to students or other researchers: 

Richard Simon: Resampling Strategies for Model Assessment and Selection

http://link.springer.com/chapter/10.1007%2F978-0-387-47509-7_8

I dislike pushing my own stuff too much, but I have written about this, too, and IIRC the Simon paper is still not freely downloadable:

(disregard in the the paper that the intro points into a somewhat different directions, the methods that are explained still apply and there are also many further refs in the paper you might want to look up)

B. Bischl, O. Mersmann, H. Trautmann, and C. Weihs. Resampling methods for meta-model validation with recommendations for evolutionary computation. Evolutionary Computation, 20(2):249–275, 2012. 

Avail here:

https://www.statistik.tu-dortmund.de/bischl_publikationen.html

And thanks to Wolfgang for namedropping my R Software ;-)

(which also allows to perform tuning with nested CV without large effort in coding)

Best, Bernd

Dear professors

Many thanks for your precious remarks. 

I used the nested CV and I found it very useful and practical in the context like mine. Because I think using this my evaluations on test set are more general and strong. So I'm using this method. In a research project, I'm trying to create a one class classifier using classifiers like ANN (auto encoder), SVM, simple gaussian and etc. Some of these methods such as ANN, KNN, Kmeans, minimum spanning tree data description  have a good performance on test set but other classifier like mixture of gaussian, simple gaussian, 1NN, Parzen density estimator still suffer from overfitting problem. 

Although methods like minimum spanning tree have a very promising efficacy in my application but I'm intrigued if I should (I can?) fix the overfitting problem of other methods or I should at least  justify that issue.  

Final point is that I've used the dimensional reduction methods like PCA and it reduce the overfitting problem but not much.  

Best,