I am looking for some citations for thumb rule for is there any thumb rule for I²-heterogeneity test in meta-analysis. Can anyone suggest some studies?
When I was working for NICE we had the rule that an I2 >=50% indicated serious heterogeneity and an I2 >75% indicated very serious heterogeneity. These thresholds are, of course, completely arbitrary, but they have a certain sense to them. Thus if a meta-analysis had an I2=50% we would carry out sub-group analyses, and, if none of these led to a resolution of inconsistency, we would present the overall effect using a random effects model. We were encouraged to be flexible with this approach, however. For example, if all studies in the meta-analysis were very imprecise, then wide disparities in point estimates might not be picked up by the I2 value, and so we should regard such pooled effects as imprecise despite a relatively low I2 value. My feeling was always that this was perhaps an incorrect approach - after all, by definition, the wide CIs in the constituent studies could be the explanation of the variability in point estimates, rather than innate differences between studies (true heterogeneity).
As for citations, I'm not sure. You could look at the NICE handbook which may be a good source. http://www.jclinepi.com/article/S0895-4356(11)00182-X/pdf is also useful reading, but does not support the use of rules-of-thumb.
Hi Kaushal,
Mark provides a nice overview how how to incorporate I2 into the review process. In terms of citable sources GRADE offer this rule of thumb (see link)
While determining what constitutes a large I2 value is subjective, the following rule-of thumb can be used:
< 40% may be low
30-60% may be moderate
50-90% may be substantial
75-100% may be considerable
GRADE also offers free software that automatically generates summary tables for meta-analyses and presents outcomes alongside risk of bias judgements which considers a range of factors including I2. I've added a link to the software below too.
http://gdt.guidelinedevelopment.org/central_prod/_design/client/handbook/handbook.html#h.g2dqzi9je57e
http://tech.cochrane.org/gradepro
As such there is no rule or it can be suggested of any cutoff value above which I2 be taken as a measure of 'true heterogeneity. As the range is 0-100%, it is wise to take >50% as acceptable and >75% as a sufficiently large heterogeneity.
Conceptually I2 is the opposite of Hunter and Schmidt's percentage of variance explained. They used 75% or more as a rule of thumb to indicate an estimate that is likely homogeneous. So the comparable cutoff for I2 would be that less than 25% indicates homogeneity.
- Cochrane Handbook for Systematic Reviews of Interventions version 6.0 has given the following estimate:
- 0% to 40%: might not be important;
- 30% to 60%: may represent moderate heterogeneity;
- 50% to 90%: may represent substantial heterogeneity;
- 75% to 100%: considerable heterogeneity.
- Reference:
- Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Available from www.training.cochrane.org/handbook.
Hi, in a multilevel metaanalysis I had a total heterogeneity of 63% but after eliminating one article with a size effect considerably larger than the rest it went down to 0.000004. Is that possible? I am worried I may be doing something wrong. Any suggestion or observation is welcome. Thanks in advance!
What does the plot look like and how many articles were remaining?
When interpreting the Cochrane rules for I2 it's also important to look at the question being asked and the sample size.
Hi Janice Montbriand , thanks for the quick reply. I have 12 studies with 58 correlation coefficients, and some belong to the same participants, that is why I am fitting a multilevel analysis. I wanted to test if the association between two variables was moderated by a third (which it it not). I used cook's distance to identify outliers. By removing this study I am left with 11 studies and 50 effect sizes. I am not sure if I can use a forest plot in multilevel analyses.
If the I square > 75% then it is recommended to test a moderating (sub group) analysis. Please see this paper for example:
Article A meta-analytical review of antecedents and outcomes of firm...
Good Luck.
Thresholds for the interpretation of I2 can be misleading, since the importance of inconsistency depends on several factors. A rough guide to interpretation is as follows:
- 0% to 40%: might not be important;
- 30% to 60%: may represent moderate heterogeneity*;
- 50% to 90%: may represent substantial heterogeneity*;
- 75% to 100%: considerable heterogeneity*.
(training.cochrane.org/handbook/current)
I² is the main source of information about the extent of heterogeneity in a meta-analysis. As soon as I² is more than 25%, the outcome should be considered heterogeneous.
- Badges
- Science topic
- Similar topics
- Phytochemistry
- Extracts