Hi Caroline,

First, on free meta-analysis software, here

http://cran.r-project.org/web/views/MetaAnalysis.html

is the R task view for the couple of dozen meta-analysis packages written in R. meta, rmeta, and metafor seem pretty popular in what I have read.

Second, I assume you are trying to estimate the population effect size for the difference in post-pre. Do the papers you cite have this mean difference but not the SD of this difference? You might need to estimate the SD from the other statistics, but this would seem more straight-forward.

Third, are gain scores all you are interested in? Often it is preferable to predict post from pre (see references on Lord's paradox, particular Wainer (1991, Psych Bull). For other papers see references in the linked paper). Might be worth comparing the results from these in your meta-analysis.

Dan

Article Comparing groups in a before-after design: When t test and A...

Hi Dan,

Thank you for your response, and the article you attached is very interesting.

R looks like it would be perfect, however, I'm unfamiliar with the programme and believe learning the computer 'language' for it could be problematic, given I have no experience with different computer languages. I'm a researcher in occupational psychology, so use SPSS most of the time.

My papers don't state the pre-post mean diffs or SDs, though the former can obviously be easily computed. The latter, I thought, were reliant on knowing the pre-post correlations also, but these are also not reported. None of the studies report t-tests based on the difference in gain scores (any t-test results reported are based on differences between groups' final results) and results from repeated measures ANOVAs / MANOVAs vary based on the number of time points which have been taken into account and so I'm not sure the results of these are comparable. I'm not sure how I would estimate the mean gain score SDs from these statistics?

Thanks again for your help,

Caroline

R is really useful, but it has a different type of interface from SPSS and most people say is more difficult to learn. I'll put a brief intro on it below (I used to use it and SPSS when I taught psych students in the UK) and lots of info on http://www.r-project.org/other-docs.html. (and there will be questions on ResearchGate asking for info on R).

As far as the measures, what are the stats that are in most of the papers?

Chapter A very brief introduction to R (ch. 1 of Wright & London, 2009)

Hi Dan,

Thanks for your quick response. I will look into the links, thank you.

As for the measures, the only ones which are consistent across the studies are the pre-post means, pre-post SDs and pre-post sample sizes. The others are a range of ANOVA and MANOVA results indicating F and p values, some of which include df, some of which don't, and some of which include eta sq / partial eta sq and some of which don't. Not all the p values are exact. Hence, and given the non-randomised design of several of the studies, I decided my best bet was to use the pre-post means and SDs to guage the effectiveness of the interventions but with no pre-post correlations reported I ran into the issue of computing mean gain score SDs, which RevMan (which had been suggested to me) doesn't appear able to deal with. Hence my foray into looking at estimations of r based on Rosenthal's (1993) work...

Any thoughts / suggestions would be much appreciated, thanks.

Caroline

It is a pain some authors don't report the sds of the gain scores. And if the studies are old they may not have those values handy.

Thanks Diana, I will take a look. I've also been experimenting with a free trial of Comprehensive Meta-Analysis which is very easy to use. It will be interesting to compare it!

And Dan, I agree, it is frustrating that the stats for meta-analysis are not all reported by authors. My studies are very recent, actually, but it doesn't seem to be the norm, at least in my field, to report gain scores and their SDs. Still, thanks for your help, I'm much clearer now about a way forward anyway. 

There is additional software for R, which makes it easier to use if you're just getting started: Deducer.

It makes the interface more user-friendly. One could compare it to SPSS. You can click on buttons etc.

http://www.deducer.org/pmwiki/pmwiki.php?n=Main.DeducerManual

That's useful to know, thank you, Lukasz.

You may have a look at:

http://davidakenny.net/meta.htm

http://www.danielsoper.com

best regards,

JP

Thanks Donna, that resource has indeed been very helpful.

All the best,

Caroline

No worries, I hadn't noticed! :-)

Caroline

To Cohens' d you can use the excel provided by Cummings (http://www.latrobe.edu.au/psy/research/cognitive-and-developmental-psychology/esci)

MetaXL from www.epigear.com is quite powerful and free