Can try Confusion matrix and ROC-AUC Score

It isn't enough to use some single measure as sensitivity, specificity, precision, PPV, NPV, etc. To use complex criteria as AUC and F_measure also are not good. For details please read my note : Negative Results Is it Good to use F-measure on Classification Tasks?

This is a very interesting and long-established research question. You may find this article useful https://www.kdnuggets.com/2017/06/7-techniques-handle-imbalanced-data.html.

I think the main concern is the generalisation capability of the model created.

Yes a good recommendation given by Daqing Chen.

Although you have highlighted the result of matthews correlation coefficient and accuracy, I still believe that those metrics are not enough to answer your question properly. Thus, I would like to ask you:

Have you looked at, e.g., "Kappa statistic"? If not, try it out, and then post its value here to help you further.

Cheers,

Dr. Samer Sarsam

Can try Confusion matrix and ROC-AUC Score

Thank you Dr. colleagues, Dr. Samer Sarsam I get cohen's kappa= 0.55

Levani Gafrindashvili

Thanks for the clarification.

Kappa statistic as well as MCC (regardless to the accuracy) show that the predictive model is good (but needs to be enhanced). Thus, I do not recommended it to be generalized.

Cheers,

Dr. Samer Sarsam

A generalized and stable precision and recall performance would do. However you should ideally deal with the data imbalance beforehand.

https://towardsdatascience.com/having-an-imbalanced-dataset-here-is-how-you-can-solve-it-1640568947eb

Hi

I suggest you to take a look at PyCM library.

We provide a general benchmark based comparison of classification models in version 2.1.

Website : https://www.pycm.ir

Repository : https://github.com/sepandhaghighi/pycm

Article PyCM: Multiclass confusion matrix library in Python

Article Learning from Imbalanced Data