Hi all,
I want to report the P values and confidence intervals for AUC values for two different multi-label classifiers. I got the following article, which is applicable for a binary classification problem.
https://stackoverflow.com/questions/52373318/how-to-compare-roc-auc-scores-of-different-binary-classifiers-and-assess-statist
But, how can we implement this in a multi-label setting?
Kindly provide some suggestions on this?
Thank you in advance!
https://machinelearningmastery.com/report-classifier-performance-confidence-intervals/
Have a look at Full article: Article The Role of p -Values in Judging the Strength of Evidence an...
Hi Ankush
For confidence intervals, there is an R package that will do this for you. In R, install multiROC then:
library(multROC)
data(test_data)
roc_auc_with_ci(test_data, conf= 0.95, type='bca', R = 100)
You get AUC and 95% CI for each response variable.
p-values are more tricky, since isn't usually clear that you need an unpaired t-test with unequal sample size and unequal variance or something else. So I suggest just reporting the bootstrapped CI.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification