Hi Tiwari,
First of all, you need to calculate accuracy for each model. Then, you need to check whether there is imbalance learning by using specificity, sensitivity etc. but, the best way is to calculate F measure. If it is greater than 0.8, the model is adequate.
https://machinelearningmastery.com/confusion-matrix-machine-learning/
.
https://stats.stackexchange.com/questions/167504/how-to-compare-2-classifers-using-confusion-matrix
https://en.m.wikipedia.org/wiki/Confusion_matrix
Execute all the classifiers minimum 100 times and collect the metrics such as accuracy, ROC, specificity, sensitivity. Then perform t test. Now compare by p value. If p value is less than .05, then we can say performances are significantly different and minimum p value indicates the highest performance.
If you dataset is imbalanced, you should be careful with accuracy, specificity, and sensitivity. In this case you should rather use balanced accuracy, G-Mean, or/and AUC. This also holds, if you are in a multiclass scenario. Lipton et al. showed that F-Measure can show very weird behavior in this case - so you should rather not use it at all.
http://journal.frontiersin.org/article/10.3389/fncom.2014.00043/full
Pl see the ref already available for the similar question at RG
https://www.researchgate.net/post/What_is_the_best_algorithm_for_classification_task
In addition to Soumen Roy's answer, consider the Wilcoxon test
https://en.m.wikipedia.org/wiki/Wilcoxon_signed-rank_test
P.S: You may find the following paper by Janez Demsar really useful: http://www.jmlr.org/papers/volume7/demsar06a/demsar06a.pdf
I suggest studying this article:
*Ferri, César, José Hernández-Orallo, and R. Modroiu. "An experimental comparison of performance measures for classification." Pattern Recognition Letters 30.1 (2009): 27-38.
First of all there is no best algorithm that can be suggested for any problem. So the option we have is to undertake extensive experiment. Before any experiment I suggest first to make sure that the data is balanced and all the required attributes/features are included. May be we can apply also feature selection method to identify best features for classification. Then start with simple algorithms like Decision tree. If the performance (accuracy, recall, precision, ROC curve) is below expectation.. adjust data set splitting techniques such as K-fold cross validation or percentage split or bootstrapping.. still if no improvement of performance, better go for artificial neural network. again if not satisfied with performance, try to use ensemble model..
use the classification leaner app in MATLAB 2016, and select the all classifications , then it will give you the accuracy of all classifications and choice the classification which is have the best accuracy.
Pl see the material for right answer
https://stackoverflow.com/questions/2414030/choose-the-right-classification-algorithm-linear-or-non-linear
Different algorithms could be used based on the following items:
What is the prevalence of the minority class in the dataset?
How many classes are in the dataset?
What is the measurement scale of the data?
Do you need any specific criteria to meet e.g. rule-based systems or online implementation?
Do you work with medical diagnosis systems ? in this case, other criteria based on Se, Sp, Pr, and DOR must be met.
Hi Tiwari,
First of all, you need to calculate accuracy for each model. Then, you need to check whether there is imbalance learning by using specificity, sensitivity etc. but, the best way is to calculate F measure. If it is greater than 0.8, the model is adequate.
https://machinelearningmastery.com/confusion-matrix-machine-learning/
Dear Mohamad Yadegari
check this link for better understanding
https://in.mathworks.com/help/stats/train-classification-models-in-classification-learner-app.html
Dear Mahmoud!
There are no "universal" classifier, efficient for all dataset.
It depends a lot on the domain
Based on the recognition rate. Recognition accuracy =(Number of test set recognized correctly/Total number of test set data)x100.
Try ROC and Confusion matrix
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification