Hi!
I have a question.
I was looking into the research question: does mohs micrographic surgery give less recurrences and metastasis than standard excision in the treatment of cuteanous squamous cell carcinomas.
I found 2 articles which didn't compare the 2 treatments (so no relative risk), but they did give me a MA of the incidence rates (just the times it occured/study population).
(See attachment of the 2 studies and their MAs)
I have 2 questions:
- how do I know if the reported pooled effect rates are significant (by their CI?). Normally a RR is significant if their CI does not content 1. But these are incidence rates (percentages), so how do I know which studies are significant.
- Is it allowed to compute the RR myself by dividing the rate/risk of the 2 treatment? so the first study tells me the risk on a recurrence is 5.5% after SE and 2.6% after MMS. Is it allowed to compute the RR myself? So 2.6%/5.5%?
Let me know thanks in advance
Interpreting a meta-analysis of incidence rates involves looking at a few key aspects:
Effect size:
- The meta-analysis will provide a pooled incidence rate ratio (IRR). This represents the overall risk of an event (disease, injury, etc.) compared to a baseline or control group, averaged across the studies.
- It will also likely report a confidence interval (CI). This tells you the range within which the true incidence rate ratio likely falls.
Heterogeneity:
- Incidence rates can vary between populations due to various factors. The analysis should assess heterogeneity, which is the variation in effect sizes across studies.
- Common methods like I² statistic are used to quantify heterogeneity. A high I² value suggests substantial variation, and the pooled estimate might not be generalizable.
Model choice:
- Meta-analyses typically use either a fixed-effect model or a random-effects model.
- A fixed-effect model assumes all studies estimate the same true effect, while a random-effects model considers that true effects might vary across studies.
- The choice depends on the presence of heterogeneity. When heterogeneity is high, a random-effects model is more appropriate.
Additional considerations:
- Look for potential publication bias, where studies with statistically significant results are more likely to be published.
- Consider the quality of the studies included in the meta-analysis.
Here are some resources that go into more detail:
- A chapter on analyzing data and meta-analyses from the Cochrane Training: https://training.cochrane.org/handbook/current/chapter-10
- An example of a meta-analysis on incidence rates: https://academic.oup.com/rheumatology/article/60/11/4982/6263849
Murtadha Shukur Hi! thank you for your response.
"Look for potential publication bias, where studies with statistically significant results are more likely to be published."
But how do I know if a incidence ratio is significant? What is the Confidence interval which tells me it's significant?
To assess potential publication bias from the information provided, we need to look at the confidence intervals (CIs) around the effect estimates. In meta-analysis studies, CIs that do not cross 1.0 are typically considered statistically significant.
For the pairwise meta-analysis models shown in section G under the PCC heading, the random effects model has a pooled estimate of 3.0 with what appears to be a 95% CI ranging from around 1.2 to 5.7. Since this CI does not cross 1.0, the overall estimate is statistically significant.
However, examining the individual study estimates and their CIs can provide insight on potential publication bias. If there is an underrepresentation of non-significant results with CIs crossing 1.0, it may suggest publication bias wherein only studies with significant findings were published.
In section H under the MMS heading, the random effects model pooled estimate of 2.1 has a CI that appears to cross 1.0 (around 1.0 to 3.4), indicating a non-significant overall effect. Looking at the individual study estimates, we see a mix of significant (CIs not crossing 1.0) and non-significant results.
While this cursory examination does not definitively confirm or refute publication bias, the presence of non-significant study results published suggests some effort to include all findings, reducing concerns over severe publication bias. A more formal assessment with statistical tests may provide further insights.
---------------------------------------------------------------
Based on the information provided in eTable 4, we can make the following observations:
For the Standard Excision treatment modality, the incidence proportion (event rate) for local recurrence is 0.055 with a 95% confidence interval of 0.032 to 0.092. This is based on data from 9 studies and a total of 1925 patients.
For the Mohs Micrographic Surgery treatment modality, the incidence proportion for local recurrence is considerably lower at 0.026 with a 95% confidence interval of 0.018 to 0.039. This estimate is based on data from 12 studies and a total of 4585 patients. The I^2 value of 5.267 suggests moderate heterogeneity among the studies.
The AFL-PDT and MAL-PDT treatment modalities have much smaller sample sizes of 19 and 21 patients, respectively, from single studies. The incidence proportions are 0.105 (95% CI: 0.026-0.337) for AFL-PDT and 0.619 (95% CI: 0.402-0.797) for MAL-PDT.
Given the larger sample sizes and narrower confidence intervals for Standard Excision and Mohs Micrographic Surgery, we can be more confident in the precision of their estimates compared to AFL-PDT and MAL-PDT. The results suggest Mohs Micrographic Surgery has a significantly lower incidence of local recurrence compared to Standard Excision.
It's important to note that these are crude incidence proportions and do not account for potential confounding factors or effect modifiers. A more comprehensive analysis adjusting for relevant covariates would be necessary to draw definitive conclusions about the comparative effectiveness of these treatment modalities.
- Badges
- Science method