Hi everyone and thank you in advance for you answers.
I'm conducting a metaregression and it turned out that some outcomes shows no heterogeneity among effect-sizes (random-model, DerSimonian and Laird method). When I say "no", I mean not significant (p(Q) > 0.05).
Is it ever possible that I will get a significance in a metaregression coefficient (DerSimonian and Laird method, Knapp-Hartung adjustment) or is it instead totally useless to procede, even if these analyses were planned a-priori.
Thank you very much!
Massimiliano
Hi Massimiliano,
A very good question! There are some authors that claim that you should proceed with moderator analysis irrespective of the results of heterogeneity tests for a-priori moderators (Geyskens et al.). Cortina (2003) argues that you should always use multiple tests for heterogeneity and moderator analysis, including between-study variaibility and credibility intervals.
However, Borenstein et al., (2009, p189) write, for example, that "92% of the observed variance comes from real differences between studies and, as such, can potentially be explained by study-level covariates"
Overall, it seems there is no difinitive answer. Technically, you could still find (significant) moderator effects despite a non-significant result of the heterogeneity test.
Article A Review and Evaluation of Meta-Analysis Practices in Manage...
Article Apples and Oranges (and Pears, Oh My!): The Search for Moder...
Article An Introduction to Meta-Analysis
Hi Massimiliano,
In my opinion, you should proceed to meta-regressions as planned. Do not care about the statistical significance. What is important is the amount of variability among the included studies which could be explained by meta-regression which is called R-squared. This is explained in detail by Borenstein in the book "An Introduction to Meta-Analysis"
best wishes,
Ramin
I'm not sure about Ramin's answer. "what is important is the amount of variability". The heterogeneity test seems to indicate there is little to no variability among the included studies, which means there is nothing to be explained. As I wrote yesterday, Borenstein seems to suggest that if there is no heterogeneity, meta-regression is meaningless. What do you think?
Sorry for the late reply. I decided to proceed anyhow!
Thank you very much for your reply and references!
Best,
Massi
My question is this: If the effect sizes vary simply due to random error, what are you trying to explain? Whether random error can be explained through moderator variables?
Even if you are totally right conceptually, I think that the problem relies (as always...) in the known limited power of the Q test compared to those achievable instead with metaregression procedures.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis