My suggestion would be, if possible, for you to group the classes that have a good balance between them, into one major class, in order to create a class division that is more balanced in a overall. In this case, you would execute one first model for distingushing between these major classes, that are more balanced between them and afterwards, for each algorithm you would execute a algorithm for distinguishing between the classes that are inside the major class. Although it may seem unintuitive to have 2 models executing in series as the probability of a correct answer would be the result of the multiplication of the accuracy from both, in previous experience of mine, i found this to be more effective other than just trying to directly creating one major model for all classes, specially, because, by doing this we should reduce the bias in the model.

As an example, i had a similar case, of imbalance for a text classification problem, where i applied the strategy above, and used for it, a sequential neural network combined with cross validation for distinguishing between the classes. It proved in this case to be much more effective than trying to take the entire data and creating a mega model at once.

As an observation, it is well possible to use KNN as a method for performing this classification as well.

For metrics, what i've previously used are accuracy and AUC. These have been proved to be good for my projects, but there are other metrics that may as well be used.

Dear Hussein Aldor

The ROC Curve or ROC Analysis is the most often used ranking metric. ROC is an abbreviation that stands for Receiver Operating Characteristic and refers to a branch of research that examines binary classifiers based on their ability to differentiate classes.

Kind Regards

Qamar Ul Islam

I recommend recall!/ sentivity metric. Since class imbalance problem allied with class disjucts and overlap gives suboptimal classification performance. This metric will allow you to see the rate of detection for all class. ROC is too optimistic a metric to be used

It might be useful to consider the macro f1-score. That way you can obtain a score that gives all classes the same weighting while in the end being a harmonic mean of such macro precision and recall.

It might also be interesting then, given the number of classes, to either consolidate some of them together in a way that makes sense for the problem you are facing (maybe some insights from literature review) or with clustering over the classes to see if they naturally join together given a set of features you are using for classification, or from a confusion matrix to see which classes get mixed up together the most.

Then creating different models for each groups of classes to further distinguish between them.

G-mean = sqrt(sensitivity*specificity) is one simple metric for measuring performance of imbalanced classification problems.

As I feel the spectral classification would be the solution to your problem. The major and minor classes would be distinguished as subclass. Superclass and Subclass approaches with Bayesian/Naive Bayesian classifier can lead to obtaining the dominant and non-dominant classes. Chameleon algorithms reorganize the subclass and class within the class.