While classification accuracy seems to provide some insights about the effect size, as classification involves many parameters (i.e. classifier, cross-validation, averaging of samples, etc.), which can modulate its magnitude, it does not directly predict the effect size. I was wondering if there has been any studies which systematically compared the two?
This discussion seems to suggest some approach for calculating effect size using the classification sensitivity: https://www.researchgate.net/post/How_to_calculate_the_effect_size_when_conducting_a_discriminant_analysis
Here is how to assess classification accuracy after removing the effect of chance, and how to compare achieved accuracy versus chance:
- https://odajournal.com/2017/04/18/what-is-optimal-data-analysis/
- https://odajournal.com/2020/06/11/generating-novometric-confidence-intervals-in-r-bootstrap-analyses-to-compare-model-and-chance-ess/
Here is how to correct chance-adjusted classification accuracy for model complexity, and an example:
- https://odajournal.com/2016/11/04/theoretical-aspects-of-the-d-statistic/
- https://odajournal.com/2016/09/19/novometric-analysis-with-ordered-class-variables-the-optimal-alternative-to-linear-regression-analysis/
Not sure, I am interested though. I am still interested in converting R-squared to a classification accuracy (naively and under the assumption of normal distributed residuals). Anyhow, at the end of this thread (https://www.researchgate.net/post/Can_we_convert_the_R-squared_of_a_linear_regression_to_distribution_overlap) you will find some R script with effect sizes in relation to accuracy, perhaps this might twist your creativity so you find an answer (if these exist?).
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification