Cross-validation, sometimes called rotation estimation, is a model validation technique for assessing how the results of a statistical analysis will generalize to an independent data set. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice.

See for more details: https://en.wikipedia.org/wiki/Cross-validation_%28statistics%29#Measures_of_fit

k-fold cross-validation is mostly suggested in machine learning. You can download Weka data mining software and explore.

Cross Validation is used to assess the predictive performance of the models and and to judge how they perform outside the sample to a new data set also known as test data

The motivation to use cross validation techniques is that when we fit a model, we are fitting it to a training dataset. Without cross validation we only have information on how does our model perform to our in-sample data. Ideally we would like to see how does the model perform when we have a new data in terms of accuracy of its predictions. In science, theories are judged by its predictive performance.  

There two types of cross validation you can perform: leave one out and k fold. The former may be more computationally demanding. 

I never used cross validation on Matlab, but one can do it on R manually or by using the R package rminer. 

To learn more on cross validation, you can also refer to the free ebook An Introduction to Statistical Learning by James. 

You can find a sample code and description of k-fold cross validation in below page   http://de.mathworks.com/help/bioinfo/ref/crossvalind.html

 Dear Yoyeeta,

Both ANN/SVM are statistical learning models used to recognize data patterns for classification and regression analyses. As models, both tools approximate to real results, but only approximate.

If you want to validate the results of your models, u need to perform cross-validation analyses with independent data to show how accurate those modes are.

I attach you a website where you'll find additional information of how to cross-validate your models with Matlab.

Best regards,

M

http://es.mathworks.com/discovery/cross-validation.html

It is mainly used to estimate any quantitative measure of fit that is appropriate for both data and model.

Cross validation approach to report the result assures unbiased result. In cross validation approach the data used for training and testing are non-overlapping and there by test results which are usually reported are not biased.

separate your data set into two subsets. One subset you use for training and other for testing. Now, do the exercise again by swapping the data sets . Report the average test result. This is call 2 fold cross validation. Similarly if you divide your entire data set in to five sub sets and perform the exercise five times and report the average test result  then that would be 5 fold cross validation.

The purpose of using cross-validation is to make you more confident to the model trained on the training set. Without cross-validation, your model may perform pretty well on the training set, but the performance decreases when applied to the testing set. The testing set is precious and should be only used once, so the solution is to separate one small part of training set as a test of the trained model, which is the validation set. k-folds cross validation is commonly used, it can still work well even the volume of training set is small.

In matlab, one way to perform cross validation is to re-arrange the training set into a random order, and select the first 20% of the sequence as the validation set when k is 5. Train you model on the left 80%, tune the parameter on the 20%, repeat the process in k times, then you should get a well trained model.

Estimada Joyeeta Singha

Estoy de acuerdo con los participantes que han manifestado lo básico respecto a la utilidad metodológica de la validación cruzada.

Considero como investigador biomédico y clínico, que la validación cruzada es la técnica del método bioestadístico cuyo propósito u objetivo es el de analizar y evaluar los resultados de una investigación, a fin de asegurar (dentro de lo probable) que los resultados obtenidos son independientes al comparar los grupos de investigación; experimental vs control (o testigo).

Por ejemplo, en el caso de investigación de la eficacia y seguridad de los medicamentos para tratar alguna enfermedad en los pacientes, los diseños factoriales, doble ciego y cruzados, tienen la finalidad de identificar el efecto de un medicamento cuando se compara con un placebo (que simula ser un medicamento pero que no lo es) y que este efecto sea independientemente del efecto placebo inherente en todos los medicamentos.

Esta técnica también se puede utilizar en la investigación educativa para evaluar el rendimiento académico de educandos sometidos técnicas didácticas pasivas tradicionales vs las que promueben la participación. Asimismo en investigación biomédica básica y en investigación en modelos de aprendizaje mediados por instrumentación de la inteligencia artificial.

Atentamente

Dr. José Luis García Vigil

Cross validation is useful for the following points:-

1-To verify the error performance of the system or model

2- To validate your training data set via validating each sub set.

Regards

Tarik

Dear Joyeeta Singha

I agree with participants that have expressed basics for methodological usefulness of cross-validation.

I think as biomedical and clinical research, which is cross-validation technique biostatistical method whose purpose or objective is to analyze and evaluate the results of an investigation, to ensure (within probable) that the results are independent of the compare research groups; experimental vs control (or control).

For example, in the case of investigation of the efficacy and safety of drugs to treat disease in patients, factorial designs, double-blind cross, they are intended to identify the effect of a drug when compared to placebo ( that looks like a medicine but is not) and that this effect is independent of the placebo effect inherent in all medications.

This technique can also be used in educational research to evaluate the academic performance of students undergoing vs traditional passive teaching techniques which promueben participation. Also in basic biomedical research and research on models of learning mid implementation of artificial intelligence.

Sincerely

Dr. Jose Luis Garcia Vigil

Dear Joyeeta Singha

Continuing our conversation about the usefulness of cross-validation, add the following:

Cross-validation can also be used in mathematical modeling and the development of artificial intelligence algorithms to analyze biomedical data emanating economic research, clinical, statistics and education, to name a few applications.

Also worth mentioning, that the statistical processing of information can be automated for micro-computer programs, developed "ad-hoc" for the kind of research that is being developed. To this end, can also be used and validated statistical programs (STATA, EPI-INFO, EPISTAT, SPSS, XLSTAT).

So far I have not used in the estadísico MATLAB data analysis of my research, but I understand that is a powerful and updated tool, developed with the methodology and technique of artificial intelligence. Processing speed exceeds languages common programming (C, C ++, Fortran) and requires the use of the latest hardware like Pentium microcomputers IV or higher, RAM of 1 or more gigabytes and advanced graphical display to 2D and 3D.

In view of the above, MATLAB has many applications such as processing large amounts of data and multi-dimensional display, along with advanced programming technique for creating artificial intelligence programs, econometric or mathematical modeling, in automotive engineering and robotics etc.

In medicine it can be used in pharmacoeconomic analysis modeling and cost-effectiveness and quality of life in the evaluation of medicines to treat human diseases. Also in the modeling and development of new drugs, including the so-called "monoclonal antibodies" for the treatment of autoimmune diseases and neoplastic.

Best regards

Sincerely

Dr. Jose Luis Garcia Vigil

Cross validation is useful in testing the success of classification on cases that are not used for building the classification model. Therefore, it leads to more realistic estimates of how well model prediction will work on new cases.

Cross validation is also used for avoiding the problem of over-fitting which may arise while designing a supervised classification model like ANN or SVM. It is a method which can give a correct accuracy of the model.

In my understanding, the purpose of k-fold cross validation is to test how well your model is trained upon a given data and test it on unseen data. But if you use all your data for training, you will have none left for testing. Again, suppose you use 80-20 split in your data, there is a probability, that if you use the 20 you used as test in train, and some other 20 from the 80 as test your model can fit in a better way. So, for this purpose we use K-fold cross validation to make sure that each and every data point comes to test at-least once.

Cross validation is to test the model performance by training it on the training data. and the this method is useful and popular as all observations are used for both training and validation, and each observation is used for validation exactly once.

As most of the people have correctly stated, cross-validation is used to test the generalizability of the model.

As we train any model on the training set, it tends to overfit most of the time, and in order to avoid this situation, we use regularization techniques. Cross-validation provides a check on how it is performing on a test data (new unseen data), and since we have limited training instances, we need to be careful while reducing the amount of training samples and reserving it for testing purpose.

The best way to improve the performance of the system without compromising much would be to use a small part of the training data itself to validate, as it might give us an idea of the model's ability to predict unseen data.

k-fold is a popular kind of cross-validation technique, in which, say k=10 for example, 9 folds for training and 1 fold for testing purpose and this repeats unless all folds get a chance to be the test set one by one. This way, it provides a good idea of the generalization ability of the model, especially when we have limited data and can't afford to split into test and training data.

I hope it helps..!!

Cheers

It can help you assess your model's performance on your training dataset.