We work on a real medical classification problem using machine learning techniques and we have small data bases. i want to know how can i validate our results? thanks in advance
If you don’t have a test set then you need to use these classes in an application and get the performance metric of the application. So think “Why do you need these classes?” Then find the application that can be evaluated by feeding your classes to it.
You can use 5-fold cross validation to validate your classification results.
Article INVARIANT TO VOLUMES OF DATA, A FUZZY MULTICLASS GENERALIZAT...
How small is small? The number of examples is not the only important figure. The number of variables is critical too.
And following Gérard Dreyfus answer, there exist also methods to increase the data size. See, for example, these two publications:
Article A New Method to Generate Artificial Frames Using the Empiric...
Article A Novel Deep Learning Approach With Data Augmentation to Cla...
Hope it helps!
k-fold cross validation techniques, kappa static analysis
Labib Sadek Terrissa
The use of modern Machine Learning methods may (wrongly) suggest that this problem is new, but I believe that the best answers for your question were provided almost a century ago. Please consider reading the second chapter of the book 'The Design of Experiments', by R. A. Fisher (THE PRINCIPLES OF EXPERIMENTATION, ILLUSTRATED BY A PSYCHO-PHYSICAL EXPERIMENT). Although it was written in 1935, I believe that if you mentally replace the lady-tasting-tea with your Machine Learning algorithm "tasting" your medical data, the two problems become analogous, and the effects of small samples (small data bases) can be properly tackled with old statistical approaches, as explained in that book.
Generally four measures or metrics used for classification task.
Accuracy , Precision, recall , F-Measure.
- Badges
- Science topic