I'm working on a meta-analysis and I'm having trouble deciding whether to use a random effect or a fixed effect model based on significant heterogeneity between studies. What p-value should I use to declare significant heterogeneity?
The decision to use random-effects or fixed-effect model SHOULD NOT be based on the detection of statistically significant heterogeneity when the collected data from included studies are combined. Doing so is a wrongly "post hoc" approach or, in a way, creates a problematic "circular justification" devoid of context that should stem from the reviewers' understanding of the phenomemon/outcome/intervention that they are studying. Choosing one model over the other should be based on how you think, beforehand, the potential differences in the effect estimates between studies exist. While the result from both models will exactly be the same when the tau-squared is 0, the use of fixed-effect model in the presence of substantial heterogeneity is possible if this can be justified sufficiently (which I think would be difficult). Please read: https://www.meta-analysis.com/downloads/Meta-analysis%20Fixed-effect%20vs%20Random-effects%20models.pdf.
For introductory purposes, statistically significant heterogeneity is considered when the the Chi-squared test for heterogeneity p-value is /= 50%. Again, do not use these thresholds to decide on the model to use.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis