I am currently working on a dataset that has PSD values of EEG frequency bands. I have modified the feature space by adding EEG frequency power ratios into the feature space. I am trying to predict the emotion Confusion using the dataset. This follows a binary classification approach.
I have conducted feature selection by selecting the features that linearly corelates with Confusion emotion (target variable). However trained SVM, KNN, Naive Bayes models show very poor accuracy (0.4 to 0.5) range. I used Ensemble ML algorithms XGBoost and Random Forest. The accuracy increased upto 0.6. I tuned the hyperparameters as well for RF. Still the accuracy didn't surpass 0.7. I even used genetic algorithm for hyperparameter optimization, still accuracy did not increase.
I would like to know what more can I do to increase the accuracy.
Thanks in Advance
You can tune the Parameters or change the algorithm KNN to Sklearn its best for Binary Classification
You can look for the behaviour of those algorithms by incorporating nonlinear data during feature selection. All these algorithms have good default parameter values so, please check for the data if accuracy is not sufficient. Sometimes, the quality and quantity of data are very vital.
What is the nature of the predictor variables continuous or categorical? How many observations "samples" do you have? How many "Confusion" = 1 and "Non-confusion" = 0 do you have? For example, you have 500 observations of "Confusion" and 100 of "Non-confusion".
Accuracy is often a poor measure of model-performance. If a balanced prediction of 1/0 is optimal, maximising Cohen's kappa is a potential choice. Yet, this enterely depends on the goal of the model. For example, true negatives could be a better choice if you would like to detect a very rare but deadly-disease. You likely accept having lots of false positive prediction during a first screaning, to make sure you do not mis anyone with the potential with the rare-deadly dease, but rule out obvious cases of not having rare-deadly diseases. Otherwise, accuracy is often not an optimal choice, for example you can have 900 times "Confusion" and 100 times "Non-confusion", then prediction of everything "Confusion" with a "lazy-model" would result in a 90% accuracy, yet the model does not really predict. Moreover, if prediction is the goal with only a few predictor variables often tend to perform marginally. If you only have a few predictor variables a binairy logist regression with a reasonable cut-off could do the job perfectly (and as a bonus, the parameters are easily shared).
If some of the words used are unclear or you would like to know more consider the introductory book on machine learning https://www.statlearning.com/ (the book is for free and easy: https://hastie.su.domains/ISLR2/ISLRv2_website.pdf).
You can read section 3.1. Pre-Processing of EEG Data of this paper which gives some more innovative ways to further improve the accuracy by pre-processing the data:
https://www.mdpi.com/1424-8220/23/7/3474
- Badges
- Science topic