The AUC value lies between 0.5 to 1 where 0.5 denotes a bad classifer and 1 denotes an excellent classifier. Is there any quantitative value for the AUC in order to segregate the quality of a classifier to be bad, fair, good, very good or excellent....or something like this?
I have attached a link which I was able to find. Please provide journal articles or documents if available.
Thanks in advance.
http://gim.unmc.edu/dxtests/roc3.htm
It depends on the context. In medical diagnosis, very high AUCs (.95 or higher) are sought.
In applied psychology and prediction of future behavior, with all the factors that can influence, AUC values of .70 and higher would be considered strong effects. See example of study on AUC and effect size in applied psychology.
Article Rice ME, Harris GTComparing effect sizes in follow-up studie...
Nos permite establecer puntos de corte de variables cuanitativas y comparar dos o más variables cuantitativas. Esta evalua sensibilidad frente a 1 - especificidad. Entre más cercano este a 1 mejor es el rendimiento con IC 95% mayores de 0.5. Un valor de área bajo la curva ROC por debajo de 0.5 habla de mal rendimiento.
As Michael says, it depends on context. I would add, though, that it is not the AUC that matters in many clinical contexts. For example, troponins for identification of acute myocardial infarction are often very high - but what matters to the Emergency care physician is high sensitivity (ie minimum false negatives) (typically 99%). ie the evaluation of a troponin is best investigated by simply looking at the cut-offs that give high sensitivity rather than the AUC. In my opinion the AUC only has value for small studies or where there are not pre-existing risk models (there are better statistics to see if whatever is being measured "adds value" to existing models). This all means that although some authors talk about "good" or "very good" AUCs (& "diagnostic" if the lower bound of the 95% confidence interval is not
Particularly I believe that the definition of a good value AUC is relative and not absolute. For example, you are comparing the effectiveness of his method and got a AUC of 0.80 (reasonable, not great), but all other work related (if any), have an AUC of 0.70. Then undoubtedly its methodology is great compared to the others. If there is no other study to compare, then your AUC is unique and by itself is great.
I found a conversion excel table that converts Cohen's d, AUC, correlation coefficients r into each other. If you are familiar with d and r you may gain intuition by playing around with this:
https://www.google.de/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&ved=0ahUKEwinubK88ajPAhUlJ8AKHX1eBMUQFgg5MAI&url=http%3A%2F%2Fwww.stat-help.com%2Fspreadsheets%2FConverting%2520effect%2520sizes%25202012-06-19.xls&usg=AFQjCNGGELYb6QFKz6K0uozfa41Yf5TOng&sig2=MZoxOH984uSZCJ_nLz5Aqg
The larger the AUC better but there is not an absolute value. However, AUC below 0.50 indicates a set of random data values which are not able to distinguish between true and false. Usually the AUC is used in a comparative way, for example: if your AUC is 0.91 and the AUC of another method (gold standard) is 0.94, for more than its AUC is great, it is still lower than Golden pattern.
Just a little addendum to Edson... 0.5 is random (= to a coin toss) not "below 0.5". Below 0.5 is simply that the opposite direction of the values of what are being measured may be predictive/diagnostic. Eg in my own field if I assess a measurement called eGFR for association with mortality I may get an AUC of 0.3. ie low eGFR is bad. Note. This is equivalent to 1/eGFR having and AUC of 0.8.
One of the problems in psychological risk assessment has been the widespread promotion of the idea that an AUC of .7 is somehow good. The reality is that it is still only moderate and authors of the many risk assessment tools are only misleading practitioners by suggesting that these estimates support the validity of the tools being used. But it is largely self-serving (and income generating) and since the AUCs are sample estimates they have little utility in individual risk assessment, because nobody can predict what an individual will do in the future. Since the model is linear, and the risk tools are largely non-parametric, and not unidimensional then most of the logic and the statistics applied are potentially flawed. As George Box once said, All models are wrong.
University of Nebraska Medical Center used the following test scores while
Interpreting Diagnostic Tests as stated below:
"Accuracy is measured by the area under the ROC curve. An area of 1 represents a perfect test; an area of .5 represents a worthless test. A rough guide for classifying the accuracy of a diagnostic test is the traditional academic point system":
- .90-1 = excellent (A)
- .80-.90 = good (B)
- .70-.80 = fair (C)
- .60-.70 = poor (D)
- .50-.60 = fail (F)
- http://gim.unmc.edu/dxtests/roc3.htm
Classification Assessment Methods
This paper introduces a detailed explanation with numerical examples many classification assessment methods or classification measures such as:
Accuracy, sensitivity, specificity, ROC curve, Precision-Recall curve, AUC score and many other metrics. In this paper, many details about the ROC curve, PR curve, and Detection Error Trade-off (DET) curve. Moreover, many details about some measures which are suitable for imbalanced data are explained.
Your comments are highly appreciated.
https://ac.els-cdn.com/S2210832718301546/1-s2.0-S2210832718301546-main.pdf?_tid=20353999-8c84-422a-8191-fd1059469272&acdnat=1536185805_03b19e159f0675a01f35512961973eb5
Presentation of the paper is here
https://www.researchgate.net/publication/327403649_Classification_assessment_methods_a_detailed_tutorial
I found this great paper for all interested, it states that: " In general, an AUC of 0.5 suggests no discrimination (i.e., ability to diagnose patients with and without the disease or condition based on the test), 0.7 to 0.8 is considered acceptable, 0.8 to 0.9 is considered excellent, and more than 0.9 is considered outstanding "
Mandrekar,Jayawant N.
Receiver Operating Characteristic Curve in Diagnostic Test Assessment
Journal of Thoracic Oncology, 2010, 5, 9, 1315-1316
Hosmer & Lemeshow (2013). Applied logistic regression. p.177:
"So, what area under the ROC curve describes good discrimination? Unfortunately there is no "magic" number, only general guidelines. In general, we use the following rule of thumb:
0.5 = This suggests no discrimination, so we might as well flip a coin.
0.5-0.7 = We consider this poor discrimination, not much better than a coin toss.
0.7-0.8 = Acceptable discrimination
0.8-0.9= Excellent discrimination
>0.9 = Outstanding discrimination"
- Badges
- Science topic