I’m trying to study if gender has a impact on the efficacy of a treatment.
None of the studies available indicate the pre-post mean and sd split by gender, so
a full meta-analytic approach is not possible (i.e. M vs F effect-size).
Thus, I decided to run a meta-regression with the treatment effect-size as dependent and the % of female in the group as independent variable.
However, nearly all of the studies only report pre and post means and sd’s, but not r/t/p necessary to calculate the pre-post standardized effect-size. These studies always have one or more comparison groups so only repeated measure ANOVA are usually applied.
Do you think it is correct if I run a simple regression with the raw mean difference (no standard error) as dependent variable and the % of female in the group as independent one ?
Many studies use the very same test to assess the pre-post change of the treated group so I don’t have to merge studies with different evaluations strategy. I know that in this way I miss the within-study variation but I’m still able to take into account the between-study.
Alternatively, please suggest any other strategies.
Thank you so much for your help.
Thank you so much Francois!
Yes, you’re absolutely right. Do you think that a weighting with the sample size or with something just derived from the sample size would be very biased?
If so, I think I will proceed using a estimate of the pre-post r according to the few studies which explicitly provide it and than performing a sensitivity analyses varying the estimate of the r to see the impact on the result.
Dear Francois,
Thanks again for your reply and suggestion!
Actually I’m pretty much in a pilot step (in order to decide whether it’s of interest to go for a full bibliographic research and complete meta-analysis of the topic), so although I know it’s not a correct way, I was wondering how much “wrongness" I could stand to get a first broad idea.
I’m going with the last approach I mentioned (estimated r and sensitivity analysis), also in this pilot step. It should be the only correct compromise.
Best,
Massi
Why not convert your measure into a measure for which you can get an effect size measure and its SE and then use the 1/SE^2 as a weight for each study? And you should also be able to create a random-effects model from this. I assume there is some variation in the operationalizations and samples, so opting for a random-effects model should be ok.
Dear Jan,
the issue is that in nearly all studies you have only the sample size, the mean (both pre and post) and the sd (both pre and post). What’s missing is the r (pre-post), the t (pre-post) and even the post - post SE. Data have been analyzed with ANOVA for multiple-treatment comparisons so they don’t take into account the single treatment ones.
With these missing data I cannot calculate a correct paired-sample effect-size and I’m stuck at early stage of the meta-regression!
Thank you for your answer!
Massi
Ah, now I see. Would it be possible for you to calculate a synethesized pre-post r across the studies from which the pre-post r can be derived and use this r as a best guess for the other studies?
By now only one paper with the possibility to calculate the r (sic). Probably with a full bibliographic search a few more will be found!
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression