Hello,
We are currently conducting a network meta-analysis of RCTs using R (netmeta package). I have relatively small number of studies.
The problem is interventions that are non-significant but has large effect size (i.e., RR 4, 95% CI [0.8,5] ) are ranking higher than interventions that are significant but with smaller effect size (i.e., RR 2, 95% CI [1-3]).
How to interpret this and how to justify this in my results/discussion?
Thanks in advance
Youssef Soliman
Network meta-analysis (NMA) is a method to compare multiple treatments by synthesizing direct and indirect evidence from randomized controlled trials¹. P-score is a measure to rank treatments in NMA based on the point estimates and standard errors of the network estimates¹². P-score is calculated as the mean of one-sided p-values that a treatment is better than the competing treatments¹². P-score ranges from 0 to 1, with higher values indicating higher ranks¹².
Non-significant interventions ranking first in P-score ranking in NMA can be interpreted as follows:
- It means that the interventions have the highest mean probability of being better than the other interventions, but this probability is not high enough to reach statistical significance¹².
- It implies that there is uncertainty or inconsistency in the network evidence, which can be due to various factors such as heterogeneity, bias, small sample size, sparse data, etc¹²³.
- It suggests that the ranking based on P-score may not be reliable or robust, and that other measures or methods should be considered to assess the comparative effectiveness of the interventions¹²³.
Therefore, non-significant interventions ranking first in P-score ranking in NMA indicate that there is no clear evidence of superiority among the interventions, and that more research or analysis is needed to draw definitive conclusions.
(1) Ranking treatments in frequentist network meta-analysis works without .... https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-015-0060-8.
(2) Chapter 12 Network Meta-Analysis | Doing Meta-Analysis in R - Bookdown. https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/netwma.html.
(3) Predictive P-score for treatment ranking in Bayesian network meta-analysis. https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-021-01397-5.
(4) netrank : Frequentist method to rank treatments in network. https://rdrr.io/cran/netmeta/man/netrank.html.