Hello dear researchers,
I want to know how should we interpret significant results of a meta-analysis while none of the included RCTs showed a significant difference?
is this because of low number of participants in RCTs? (approx. 20 to 30)
how should we discuss on this issue in our papers?
best,
Parsa
Yes, one of the main reasons for doing a meta-analysis is to increase the statistical power of overall tests, via increases in sample size. So, it is possible that individual RCTs were not sufficiently powered to detect small effects, but that pooling them together enables uncovering small effects. It is difficult to make a general statement about how this 'should' be interpreted, but one possibility is that power in each RCT was too low to detect the type of effects the meta-analysis could detect. Hope this helps!
This could be due to a low number of participants in RCTs. When you combine all these RCTs in a meta-analysis, the statistical power of the overall test is increased. I suggest you may also perform an egger test to identify the presence od small-study effects.
I agree with David and Sajesh, but i would look also at the effect of potential outliers. You might one or a few studies that have an unduly strong influence over your final outcome. If 9/10 studies show no or a weak effect, and 1/10 studies show a large effect leading the meta-analysis to yield a significant effect, I would still be sceptical about the actual effect.
Risk of bias is an option I think. To exclude high risk studies, sensitivity analysis should be undertaken on low risk studies. After that, pooled estimate of all studies and that of low risk of bias studies can be compared. When there is no variation between the estimates, it should be interpret as robust. If not, you should interpret with caution.
most likely, it is because of low power for each of the RCTs per se. as mentioned, there is a potential risk of outliers, I recommend depicting a funnel plot and its subsequent analysis. Moreover, you might perform some simulative run on the asymmetric funnel (showing pub bias) to impute probably biased reports that are missing in the literature and examine again for significance. the latter might not change your p-value but most likely affects your 'TE'
- Badges
- Science topic
- Similar topics
- Philosophy
- Esthetics
- Interpretation