I am using the programme "Comprehensive Meta-Analysis" (CMA) to meta-analyze the effectiveness of an intervention. The effect size measure used is Cohen's d.
Using CMA programme, there are two different categories of required statistical data for comparing two independent groups with only post-data, and those with both pre-post data. Since some of my included studies have two independent groups with sufficient pre-post data for only the experimental groups, is it justifiable for me to only input these pre-post data of the experimental groups in CMA as if these studies are matched groups with before-and-after comparison in CMA?
Second, some the built-in formula in CMA require the "mean change" or "difference in means". Is "mean change" here refers to only the change in scores between two time-points within the same group? Is "difference in means" here referring to only the difference in scores between two independent groups?
Some studies reported the difference / change in means directly, while some other studies only reported the scores (e.g. means with SD) at each time-points respectively. For example, if I calculate the different in means between post-test and pre-test manually myself, does this difference equal to the "mean change" or "difference in means" as required by CMA mentioned above?
Any advice or recommended resources would be highly appreciated. Thanks a lot in advance!
You can use different effect sizes like mean change or pre and post data and combine them in a meta analysis in comprehensive meta-analysis software. However it is not advisable to combine pre and post data from one study with post only data from another study in a meta-analysis.
Thank Nipun for the advice! However, for your advice "not to combine pre and post data from one study with post only data from another study in a meta-analysis", isnt' it what CMA software has been doing? Do you mean a meta-analyzed effect size should be separately calculated for a group of studies with pre-post data entered, and another meta-analyzed effect size for another group of studies with post-only data?
I think another major problem I am facing is, for studies with independent group design (e.g. experimental and control group), can these studies be treated as before-and-after studies (with 1 matched group only) that we only use the data of the experimental group in meta-analyzing effect size?
thanks all again
Hi Gloria,
We recently conducted a meta-analysis using CMA. Some of our studies were cross-sectional with 'post' data only, while others were longitudinal with pre and post data. We therefore conducted two separate meta-analyses: one cross-sectional and one longitudinal. The cross-sectional meta-analysis included the data from cross-sectional studies as well as the post data from the longitudinal studies, and the longitudinal meta-analysis included the pre/post data from longitudinal studies.
If I understand your second question correctly, I don't see an issue with using the pre-post data from only experimental groups.
Best of luck!
Hi Everyone,
I am not familiar with CMA, however I think I can help address some of the issues raised above. I believe Emily and Nipun are correct, these should be treated as separate meta-analyses. If I understand correctly, you have different types of effects sizes that you would be calculating, one effect size could be calculated by comparing the "mean change" in one group to the "mean change" in another group, a second type of effect size could be calculated by comparing the pre- vs. post- values of a single group, and a third type of effect size calculated by calculating some sort of a standardized difference score from the post-only (cross-sectional) data. In my opinion, CMA might produce an effect size, but these are really three separate effects, and three separate answers, to three separate questions.
To second Emily's opinion, we did the same thing in the article below, except with the control groups. We wanted to examine the mean change in body weight in college students over time, so we combined the data from prospective cohort studies with data collected from the control groups of RCTs (since all data was essentially watching participants as they changed naturally over time).
Article Change in Weight and Adiposity in College Students A Systema...
Would it help if we knew what question you were trying to answer with your review?
Cheers,
Mike
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis