I am working on a meta-analysis on effectiveness of a program. As indicators of heterogeneity in the effect size estimates, I have obtained the Q statistic and the I^2 statistic from the program Comprehensive Meta-Analysis (CMA) 3.0. Now I am using the formula on page 123 of Borenstein, Hedges, Higgins, & Rothstein (2009) to compute the 95% confidence intervals of the I^2 statistics. But some problems occur.
1) When there is only 1 study (k=1), should we (or can we) still compute the Q and I^2 statistic? Because I still obtained 0.00 for both Q and I^2 in CMA. However when I go ahead to compute the 95%CI of I^2, it cannot be computed. I suppose heterogeneity issue would not be irrelevant when k=1?
2) Is it possible and reasonable for the lower limit of the 95% confidence interval of I^2 be negative values? The range apparently seems too large...? What does a negative value of I^2 imply?
The 3 problematic cases for this question:
Q=5.708, df=4, I^2=29.92 [-81.34, 72.92]
Q=3.527, df=1, I^2=71.65 [-26.04, 93.62]
Q=1.17, df=4, I^2=0.00 [-1646.34, 33.07]
2) Another problematic case is that, the computed lower limit of I^2 is larger than the point estimate of I^2 statistic. Does this imply my computation error or other possible issues?
Q=117.220, df=12, I^2=71.65 [84.34, 93.31]
the reference is:
Borenstein, Hedges, Higgins, & Rothstein, 2009
http://onlinelibrary.wiley.com/book/10.1002/9780470743386
3) Regarding the article below, may I know if anyone could share with me, in simpler ways, how to conceptualize, conduct, and interpret the testing of potential publication bias alongside moderators such as PET-PEESE in this article? I am required to conduct this test in my meta-analysis but I don't quite understand the simulation analyses in this paper.
Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods, 5(1), 60-78.
Much thanks for your help in advance! Thanks
Calculating heterogeneity statistics for k=1 is obviously meaningless and not really possible
Meta-analyses with very small number of studies can lead to very wide CI for I squared (in addition to being a biased measure in these sort of meta-analyses) which decreases its usefulness.
Please see
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4410499/
This might interest you regarding PET/PEESE analysis and its' weaknesses:
http://datacolada.org/59
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics