Health Care Quality Measurement may involve making measurement tradeoffs. The attached presentation on AMI quality measurement describes some of these tradeoffs and some approaches to handling them. What has your experience been in dealing with these kinds of measurement challenges?
Data Sharpening The Quality Measurement Blade - Slides & Data
We struggle with this a lot in developing quality indicators, especially because we most often rely on hospital administrative claims data, which lacks much of the clinical detail that ideally would be nice to include in measurement. So already in using claims data we've made a trade-off sacrificing the clinical detail found in the medical record for the decreased time and cost of using readily-available claims data. As we develop indicators, we think about the intended purpose (quality improvement, public reporting, pay for performance, research) and how various trade-offs in sensitivity and specificity will impact use of the measure for these various purposes. Sometimes internal validity is more important than generalizability, so then more precision is needed at the cost of more stringent criteria and smaller samples. Other times its more important that a measure be generalizable, and interpretable, even when we know some specificity will be lost. We constantly emphasize thinking about the aggregate rate (for an area, hospital, health plan, etc) rather than individual cases. And above all, we keep in mind this principle: Don't let the perfect become the enemy of the good.
Yes, these are classic considerations when designing measures for diverse purposes. I was struck early on by Dr. Deming's statement that all measurement is a process: change the process, you change the measure.
As for claims measure trade offs, when claims data was all we had to work with, we made the most of it by augmenting claims data sets with other data, and did fine for the purposes (quality improvement) at hand.
Thank you for sharing, and if you have other details, I would enjoy learning more about your work.