What methods are there to create new cutoffs for a given continous measurement as an indicator of health, or diagnosis of a disease? For example, how can I create new cutoffs for C-reactive protein as an indicator of severity of a disease (normal, mild, severe....)? To be specific, I am interested to know the methods for new cutoffs, not about the C-reactive protein per se.
Hello Henock,
By generating a ROC curve you could decide a clinically valid cutoff. It is generated using sensitivity and 1- specificity and plotted for different cutoffs. Hence this displays the tradeoff we make. The "shoulder" of the ROC can be used as the clinical cutoff.
Hope this helps.
Anuji
You would first need to find out what the concentration of the marker of interest in a 'normal population' is (eg healthy individuals). You then need to measure the concentrations of that marker in 'sick' individuals - and establish the range. You can then subdivide that group by disease severity.
First you will need to establish the concentration of the marker of interest in the 'normal population' (ie healthy individuals).
First you will need to establish the concentration of the marker of interest in the 'normal population' (ie healthy individuals).
by measure the concentrations of that marker in 'sick' individuals - and establish the range. You can then subdivide that group by disease severity.
What methods are there to create new cutoffs? - ResearchGate. Available from: https://www.researchgate.net/post/What_methods_are_there_to_create_new_cutoffs [accessed Nov 2, 2015].
As far as I know, there is generally no formal research method used to produce cut-offs. Earlier responses to your question give some ideas on what may play a role. But in the end I think that cut-offs are set in e.g. guideline committees, which are usually guided by empirical evidence and the cut-offs that happened to have been used in reporting research results before clinically applied cut-offs were set.
I would say that, at least theoretically and probably with a health economics slant, it seems best have the cut-off depend on development of technology: First determine what will be done based on the test result, what will be the effects of applying different cut-offs, balance the favorable and unfavorable effects (which may or may not include costs) at different cut-offs and select the optimal cut-off. With e.g. a new treatment becoming available, the expected effects of intervention after a positive test result will change and therefore the optimal cut-off may change.
As Ishag Adam I think that ROC analysis will help to find the best cutoff, because at each point of your measure (in your case C-reactive protein) it shows the combination of sensitivity and (1-)specificity.
With the additional explanation of Bea Tiemens, I think most important issues have been raised now:
- you need a large enough dataset in a normal and in a "diseased" population (which requires a gold standard for the diagnosis of the disease)
- this will allow you to calculate the sensitivity and (1-)specificity for different possible cut-off values, and visualise them in a ROC curve
However, choosing the "best" cut-off then still depends on the context you are looking into (e;g. the relative weight of avoiding false positives/false negatives). You may need to reconsider if you even need a cut-off: translating your continuous measurement into the probability that the disease is present, may be much more informative.
Have a look at http://stats.stackexchange.com/questions/29719/how-to-determine-best-cutoff-point-and-its-confidence-interval-using-roc-curve-i and other sources.
http://stats.stackexchange.com/questions/29719/how-to-determine-best-cutoff-point-and-its-confidence-interval-using-roc-curve-i
Hello Henock,
By generating a ROC curve you could decide a clinically valid cutoff. It is generated using sensitivity and 1- specificity and plotted for different cutoffs. Hence this displays the tradeoff we make. The "shoulder" of the ROC can be used as the clinical cutoff.
Hope this helps.
Anuji
If you have a reference standard, you can combine the ROC curve with the Youden test to decide the better cut-off point. However, you may be interest to favour more sensitivity than specificity or vice versa depends on the aim of the exam (e. g., screening or confirmation).
Hi Henock,
A similar question and valuable responses on the same topic you raised has been discussed here on RG almost a year ago. See https://www.researchgate.net/post/What_are_the_methods_to_determine_cut-off_points_from_an_ROC_Analysis
However, two options you have is to apply Receiver Operator Characteristics (ROC) curve analysis or the Optimal Discriminant Analysis (ODA) model to derive a cut-off that accurately predict/discriminate your outcome variable of interest (say normal or severe).
Most traditional software like SPSS, XLSTAT, STATA, R....etc runs the ROC analysis, but for the ODA model, you would need the UniODA software (quite new, but powerful, especially when you have a weighted data).
some useful assumptions are discussed in the link I provided above.
Hope this helps.
Victor
Sorry, I forgot to mention. Another issue I have noticed in data driven cut-off points (like the ROCs and co) is the possibility of instability. When such instability is noticed, the cut-off point that correctly classify those with severe category of your outcome disease of interest may NOT always 'optimally' classify those with or without the disease. So it is always of interest to validate the cut-off point derived through either an ROC analysis or ODA model.
However, I am not sure if such validity tests exist for ROC, but I found that the ODA model runs exhaustive cross-validation such as the leave-one-out (LOO) to test the validity/stability of the derived cut-off point.
But ultimately, in medicine, utility-based approaches should be the paramount checkpoint, i.e whether sensitivity is of interest or specificity.
Cheers :)
Thank you so much all for your disposal in sharing your ideas and experiences. So far, I learnt that ROC and Discriminant Analysis (DA) are important for supporting expertise in defining cut-offs. I am gonna use of these methods to do so. By expertise, I mean it's some times tricky to entirely rely on stat/mathematical models unless we validate them with a body of knowledge of the field we are health with.
@Maduabuchi Oguoma, I know that STATA can execute discriminate analysis. Is ODA, what you are referring to, different from DA?
Thank you all again.
Dear Henock Yebyo
Yes any of these soft ware can run them for you with ease
Wish you a good luck
Hi Henock,
ODA model is different from Fisher's Linear Discriminant Analysis which is found in most traditional packages like STATA. ODA paradigm is only found in UniODA software.
If you are unable to lend its services, then ROC would be your next best bet. However, the need to also assess the stability of your derived cut-off point that discriminate/classify/predict severe condition is important.
Another thing I noticed is that in your question, your outcome variable is not dichotomous (you have three steps - normal, mild, severe). Of the softwares that I have used (SPSS and XLSTAT), ROC requires a dichotomous outcome variable. But ODA handles more than two category outcome variable, giving you cut-off for each category
See Refs for ODA:
1. Yarnold PR, Soltysik RC. Optimal data analysis: a guidebook with software for windows. Washington DC: American Psychological Association; 2005
2. http://odajournal.com/
- Badges
- Science topic