Conference abstracts typically lack the comprehensive data found in full-text publications, making it challenging to evaluate their risk of bias and overall quality. How should these limitations be addressed in the review process?
Yes, the methodological quality of conference abstracts should be assessed when included in a systematic review and meta-analysis. Here’s why:
Limited Information: Conference abstracts often provide only a summary of the study, lacking details about methodology, sample size, and analysis. This can make it hard to judge the study’s validity and reliability.
Risk of Bias: Studies presented only as abstracts may not have undergone peer review, increasing the risk of bias. They may also report more positive results (publication bias).
I think the short answer is yes, we should. Conference abstracts are seriously limited. They're like the movie trailers of research. That creates a real challenge when trying to evaluate their risk of bias. The truth is, excluding all conference abstracts can introduce publication bias (since negative results often never make it beyond the abstract stage), but including low-quality studies can introduce different biases. It's a balancing act. What I've seen work well is contacting abstract authors for additional information when possible. Sometimes they have unpublished data they're willing to share that can fill in the gaps.
Since conference abstracts often provide limited methodological detail and rarely undergo full peer review, their inclusion in systematic reviews requires a cautious and transparent approach. I agree with Dr. Ricolfi that it is an effective strategy to reach out to the authors for supplementary data or clarifications that are not available in the abstract. Additionally, applying a tailored risk of bias framework and clearly distinguishing abstracts from full-text studies in the analysis helps maintain analytical integrity. Including abstracts in sensitivity analyses can also prevent them from skewing results, especially given their tendency to report early or favorable findings! Documenting these steps in your review ensures that their limitations are acknowledged without dismissing potentially valuable preliminary evidence.