I've recently completed a receiver operator curve analysis, where the goal is to identify the optimal cutpoint on a diagnostic test. We have two versions of the diagnostic test. The original test, and one scale where one of the six items was changed with alternate wording.
For the original scale, the AUC was .88, and the cutoff of two looks best (.82 sensitivity, .84 specificity, .66 youdens)
For the alternate scale, the AUC was .90, and the optimal cutoff looks like it was 1 (.93 sensitivity, .76 specificity, .68 youdens)
So, we are trying to decide if it is ultimately worth changing the wording of one of six items (i.e., selecting alternative scale), which looks like it may change the cutpoint of the overall test if adopted.
So, some questions:
1. Is a 2 percent increase in AUC important? Anyone have any guidelines for what constitutes an important change in AUC?
2. Would you take an 8% drop in specificity (.84 to.76), if you have an 11 percent increase in Sensitivity (.93 vs. .83)?
I look forward to your replies!
Choice of optimal cut off depends on the objective of your study. Do you require more sensitive or specific test? If both sensitivity and specificity are equally important, you should use cut off with 0.66 youdens with good sensitivity as well as specificity. Anyway, in both the scenarios, AUC is good because it is >0.7. there are many other aspects are also involved while choosing cut off such as cost. Following article may help you.
ncbi.nlm.nih.gov/pmc/articles/PMC5082211/
العدل يختلف عن المساواة، فالمساواة هي الظلم بعينه، بينما العدل هو إعطاء كل ذي حق حقه بالنظر إلى مجموعة من العوامل المحيطة بكل شخص
Thanks for asking this excellent question. Unfortunately, I am not an expert in this field. However, I believe that these journal paper may provide you with some useful knowledge. The links are:
https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.hindawi.com/journals/cmmm/2017/3762651/&ved=2ahUKEwj-v62V1Y_yAhWgILcAHTzaAjsQFjAAegQIAxAC&usg=AOvVaw0X-kLVWTRO1DJKQcEjIwyA&cshid=1627815516250
https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5082211/&ved=2ahUKEwj-v62V1Y_yAhWgILcAHTzaAjsQFjABegQIFxAC&usg=AOvVaw2W78qZkxHTu6mcrIPvpG3r&cshid=1627815516250
https://www.google.com/url?sa=t&source=web&rct=j&url=https://medcraveonline.com/BBIJ/using-the-roc-curve-to-measure-association-and-evaluate-prediction-accuracy-for-a-binary-outcome.html&ved=2ahUKEwj-v62V1Y_yAhWgILcAHTzaAjsQFjAEegQIGRAC&usg=AOvVaw2aSAzzIU0LXlzfw-TX0may&cshid=1627815516250
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis