I am doing a prognostic meta-analysis to investigate whether anion gap (a biochemical marker) can predict short-term mortality in critically ill patients. While the effect measure I am mainly interested in is odds ratio, many studies have instead reported the area under the ROC curve (AUC) as an effect measure. I am aware that AUC is a mainly a diagnostic statistic but I was wondering whether it would be appropriate to pool AUCs for this purpose?
Thanks!
Hi Stella,
As often: it depends:
Although AUCs adequately reflect the overall discriminitative ability of a test or marker, it has little value to inform about the clinical value of a test - who is likely to die and should be subjected to some intervention and who is not. However, the same holds for the odds-ratio.If your aim is to obtain this general discriminative ability of this marker, AUCs could be used as an alternative.
For clinical use, however, it would be more interesting to pool sensitivities and specificities; these are more informative how well the test can identify those who will die and/or those who will survive. But then these prognostic metrics should be reported in the original studies, which may not be the case if they only report AUCs.
Hope this helps.
Regards, Brent
Thanks Brent, they don't report sensitivity and specificity but I will pool the AUCs
The question is quite generic therefore the answer will be generic too. I cant recall reading a meta-analysis of prediction models. The first suggestion is to take a look at webpage of the Cochrane prognostic group for systematic reviews. Pretty serious investigators on this topic have hit hard on bad practices on this topic over there. My major concern on conducting this meta-analsys is the risk of combining stuff that should not bee combined. Suppose the first model is outcome ~ pred1 + pred2 + pred3 and the second outcome ~ pred1 + pred3 + pred4. As far as I concern, all prediction models tested and included different predictors in different populations. In essence this meta-analysis is combining results from different models, thus its summary estimate is not interpretable. Even if you are pooling Se and Sp, this would be like combining CT scan and MRI results for the same outcome. No wonder the external validation is recommended for all prediction models, as the prediction model later or elsewhere may have a different accuracy. So far, I believe a plain systematic review would be a more fair and interpretable result.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis