For analyzing the performance of classifiers in classification several metrics can be used such as Accuracy, Precision, Recall/Sensitivity/TPR, Specificity/TNR, F1-score, G-mean, Area under ROC curve (AUC), Area under the precision-recall curve (AUC-PR), etc. I want to understand which metric or set of metrics can precisely evaluate the classification performance while classifying the class imbalanced dataset.
It depends on your dataset. There is no general answer that works better in every practical application. However, usually AUC and F1-score work well. To give a concrete example, consider the classification of DNA strings. For simplicity consider 4-letter ones. Then if your purpose is to classify those which show mutation (imbalanced dataset), then maybe there is no need to use a specific metric; to get good results just use the accuracy.
Dear Mahit Kumar Paul
As Mostafa Kiani Shahvandi mentioned that this totally depends on your datasets. All these performance indices works exceptionally well in defining the precision and accuracy of classification model. Although based on my experience, TPR and TNR can be used as these are the best indicators for any datasets.
Hope this helps.
Regards
Dear Mahit Kumar Paul
These links might connects all the dots together, have a look:
1. https://machinelearningmastery.com/tour-of-evaluation-metrics-for-imbalanced-classification/#:~:text=Precision%2DRecall%20Metrics&text=The%20F%2DMeasure%20is%20a%20popular%20metric%20for%20imbalanced%20classification.
2. https://towardsdatascience.com/metrics-for-imbalanced-classification-41c71549bbb5
3. https://towardsdatascience.com/the-5-classification-evaluation-metrics-you-must-know-aa97784ff226
4. https://stats.stackexchange.com/questions/222558/classification-evaluation-metrics-for-highly-imbalanced-data
5. https://medium.com/datasciencestory/performance-metrics-for-evaluating-a-model-on-an-imbalanced-data-set-1feeab6c36fe
Kind Regards
Qamar Ul Islam
Dear. Mahit Kumar Paul
What do you mean by imbalanced dataset. Gould you give an example?
Because the metrics, that evaluate the performance of any system will be based on the database.
Dear Fthi Albkosh
Say, a dataset with 100 instances has two classes: class-1 and class-2 where class-1 contains 10 instances and class-2 contains 90 instances. We see the class distribution is unequal and this is called an imbalanced dataset. Here the Imbalance Ratio (IR) is IR=(#majority class data)/(#minority class data)=90/10=9. Thank you.
- Badges
- Science topic