I want to know what criteria should be used when you do an analysis of ROC curves and gets results of sensitivity and specificity.
Or will depend if we want screening criteria with my new test?
Broadly speaking there are 2 ways of defining an optimum, in line with the previous comments: a data-driven and a decision-analytic (or ‘‘utility-based’’) approach. The data driven approach is to maximize Youden’s index, which is defined as sensitivity + specificity - 1. This maximum is in the upper left corner of the ROC curve.
The decision analytic approaches takes the clinical context as the starting point.
The utility, or relative satisfaction, of the consequences of a true or false classifications are formally considered. See e.g. the OptimalCutpoints package for R software, and a review paper (https://www.researchgate.net/publication/51496344)
Article Performance Measures for Prediction Models and Markers: Eval...
The ROC curve analysis provides a crude balance of sensitivity and specificity akin to the Youden score (but not mathematically identical). Most programs generate the optimal cut off but this should be used with caution regarding application to a clinical population. Whilst the ROC does help to chose a cut point on a linear scale, for clinical purposes it is often better is to use clinical utility. This calculates false positives and false negatives and test positive and test negative rate. For example in screening a low false negative rate may be the priority, not an equal balance. This can be accessed here: www.clinicalutility.co.uk
Determine the value of test results: true vs false positive and true vs false negative, calculate the total value at each point of the ROC curve based on the values of the test results and prevalence of the condition you test for, and select the cutoff at the maximum total value.
The value that you determine can be based on only health effects and may also involve costs, depending on your perspective on how to optimize the test result.
I was also bewitched by this question for a while until I eventually understood: The very question is ill posed!! There is NO MEANINGFUL general purely statistical solution: There is ALWAYS a trade off involved between the costs for false positives and false negatives. And this is an decision on utilities that depend on the application.
Think about the concrete underlying problem and do not use omnibus general procedures unless you really understand the implied weights on the trade-off are.
Broadly speaking there are 2 ways of defining an optimum, in line with the previous comments: a data-driven and a decision-analytic (or ‘‘utility-based’’) approach. The data driven approach is to maximize Youden’s index, which is defined as sensitivity + specificity - 1. This maximum is in the upper left corner of the ROC curve.
The decision analytic approaches takes the clinical context as the starting point.
The utility, or relative satisfaction, of the consequences of a true or false classifications are formally considered. See e.g. the OptimalCutpoints package for R software, and a review paper (https://www.researchgate.net/publication/51496344)
Article Performance Measures for Prediction Models and Markers: Eval...
I am not a statistician, but I got excellent results in the past by cutting at the maximum value of the Matthews correlation coefficient.
http://en.wikipedia.org/wiki/Matthews_correlation_coefficient
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis