I tried performing a meta-analysis of single arm studies (i.e without controls) using openMeta-analyst which allowed me to combine the effect estimate in form of proportions. Different organ transplants with similar endpoint were included from various studies. I was able to obtain
1) the overall estimate of all organs
2) perform sub-group analysis to find the estimate for specific organs, timing of treatment(7 days), study design and availability of insurance cover or not
3) Did meta-regression to assess impact of covariates like timing of treatment
My PI wants a direct comparison of the point estimates from the subgroups already meta-analyzed. Is this a good practice and how can I do this without controls? I thought of using one organ say heart as the intervention and liver as control and then including by the number of events/total number of subjects for studies that provided data. For the corresponding control or intervention without values (since this direct comparison was not done in individual studies), I used zero and then corrected with 0.5 automatically which the software handles pretty well. Is this an ideal way to go in order to obtain the RR or OR across different sub-groups?
Please see below a schema of what I did:
Study organ 1 organ 2
A 0/0 6/20
B 0/0 4/9
C 0/0 8/23
D 6/21 0/0
E 34/45 0/0
F 12/50 0/0
ABC don't have information on organ 1 while DEF don't have info on organ 2. I am hoping this set up can help me unravel the difference between organs for a specific outcome measured.
I would appreciate your urgent response.
You cannot calculate odds ratios or risk ratios for single arm studies, by definition.
In this case, you can do a meta-analysis of proportions and sub-group by organ. You can then test for differences between sub-groups, which is most commonly done by effectively testing for significant heterogeneity between the two sub-groups. This is probably only reasonable in a random effects model, and may not be appropriate if you only have 6 relatively small studies.
Jack Henry Thanks for your prompt suggestion. As a follow up, how do I test for significance of heterogeneity between the estimates of subgroups? Should I use student t-test to do that or just explore the confidence interval to see if they overlap or not? Thanks in anticipation of your response.
If there are only two sub-groups, confidence intervals that do not overlap are de facto 'statistical significance'. You cannot use a t-test - most software packages calculate the chi-squared statistic for sub-group differences but the exact method implemented will depend on whether a fixed or random effects model is used. Both meta for R and RevMan (as far as I know) have this built in.
You can assess for heterogeneity by eyeballing the CI on your forest plot, but I think most Stats software will also allow you to perform the I^2 statistics or Cochrane’s Q.
However, as Jack Henry has alluded to, there is a find line between useful and useless meta-analytic methods especially when analysing single arm studies.
@Michael, thanks for your contribution. I used openMetaanalyst which provided Q, I^2, Tau^2 and CI statistics in addition to the point estimates expressed as proportions. I needed a second opinion as my collaborators wanted an analysis that is pretty difficult to perform with single arm studies.
Jack Henry Just to be accurate, I have in some cases 2 groups and at other times 4. I used random effect model for the analysis. Will the analysis of CI in a subgroup of 4 follow similar pattern as is the case for 2 groups? For instance, can I compare lungs and liver from a subgroup analysis of lungs, liver, heart and kidney?
@Victor Yes, if any two subgroups have 95% CIs that do not overlap then you can say that the difference in proportions between these two subgroups is significant. The certainty of this conclusion depends on the number of studies in the groups. If you have multiple subgroups then tests for subgroup differences tell you whether there is a significant difference between the multiple groups, or more accurately whether the subgroup estimates are significantly heterogenous. There may be scenarios, especially in cases of multiple subgroups, where the CIs overlap but there is still significant heterogeneity between groups.
In reality, depending on the number and design of studies, these tests do not really constitute conclusive evidence and you may be better off analysing the results qualitatively rather than searching for 'statistical significance'.
Jack Henry You are fantastic. I should consider a collaboration with you on my next project. You can reach me @ [email protected] or [email protected]. I will be most glad to connect with you via personal email. Thanks a zillion.
The analysis of the included study would depend on the answer you are looking to identify in the metaanalysis. Define your PICO carefully and that will shape your analysis.
When evaluating single arm study, make sure to evaluate how the outcome is defined i.e. is the outcome response vs no response or is it increased or decreased response. Lets assume in a study with a binary outcome (e.g., response vs. no response) if the objective is to identify 'any effect' whatsoever - (i.e., the null hypothesis is “zero response” or equivalently that the lower bound for the confidence interval for the response rate is greater than zero).
Can we apply a meta-analysis based on one group?
We do not have control group and treated group. We just have a treated group.
Seifollah-Seif- Gholampour its been done severally and based on my enquiry here and elsewhere, I have been able to do it. You can only report in proportions in most cases. Without a control group, RR and OR cannot be assessed. However, Wald type test can be used to compare coefficients of various subgroups to determine if there is any significant difference between them and this type of test produces Q and P values that can be reported.
You may use R (metafor), OpenMeta[Analyst] or JASP to perform meta-analysis of single arm studies. There are others out there but definitely not Revman.
Victor Ejigah
Thank you very much for your response. I do not need RR and OR. I am working with SPSS and I could not find any options to perform this. Do you have any experiences about Excel or SPSS for preforming this?
Yes. I have used SPSS but it is not meant for meta-analysis. It is better for performing descriptive statistics and, parametric and non-parametric tests in primary studies. Besides you need to subscribe to use it. You can do with JASP all you need to do with SPSS and more. JASP has modules for factor analysis (eg principle component analysis), meta-analysis including Bayesian meta-analysis and even machine learning. You don't need to know how to code to use JASP. R is also free but you need the syntax to get the job done. I actually did the Wald test with R. OpenMeta[Analyst] is also free and easy to use.
With excel, you can use Meta-essentials which is also open source. However, it is designed for dichotomous data which makes it difficult to use for single arm studies. Try the options I mentioned above and you will love them. Most often, you may need more than one software to get all your answers depending on what type of analysis you intend doing.
Generally I used the attached option on SPSS.
However, I need Forest plot. Can I have forest plot for one group (single-arm)? If yes, you think which software is open-access and faster
Good to know you have the version that can perform meta-analysis. Yes you can have forest and funnel plots with single arm studies. JASP allows you to do this quickly and you can see the plots change as you add your data. It gives a good output and allow you to perform diagnostic tests like trim and fill analysis and influence measures. For subgroup analysis and meta-regression, Openmeta[Analyst] is better. Both software are open source.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis