I am preparing a correlational meta-analysis. As a measure of effect size, I have decided to use the correlation coefficient r. Now some studies provide multiple correlation coefficients based on different measures for the predictor, the criterion variable, or both. How do I combine these different values so that there is only one effect size per study? Do I have to z-transform all correlations beforehand? I have also read that I have to account for within-study correlations among the outcome measures. However, I do not fully understand why and how to do this. Can anyone help me out?
Hi Sara,
to combine effects you need to z-transform them first. In short, you would conduct a small within-study meta-analysis. But, if all of the predictors/outcomes you include are of interest (this is why you would include them), you should opt for a hierarchical meta-analyses (multi-level). In this way you can model both within-study variance and between-study variance (effects will likely be more similar within than between studies). If you don’t do this it is likely that you over/underestimate the precision of the final estimate. There are several ways of doing this. I can send you two meta-analysis in which I have used this type of multi-level approach if you want.
Here is an example:
Article Is Bilingualism Associated With Enhanced Executive Functioni...
Sara, if you want me to give more details on how to model your type of data, you can also e-mail me (you should have my e-mail adress).
Hi Sara,
the best way is to avoid aggregating/averaging correlations and conduct a three-level meta analysis. In this approach, ES are nested in studies and you get all the estimates for true variances splitted into a within-study and between-study component.
These meta-analyses can easily be done in R's metafor and metaSEM packages.
If you allow me a self-citation, see the followign paper for further references
Steinmetz, H., Knappstein, M., Ajzen, I., Schmidt, P., & Kabst, R. (2016). How effective are behavior change interventions based on the theory of planned behavior? A three-level meta-analysis. Zeitschrift für Psychologie, 224(3), 216-233. doi:10.1027/2151-2604/a000255
Best
Holger
Hi Holger and Jan,
thank you for your precise answers. I will follow your suggestions and conduct a multi-level analysis to account for within- and between-study variance. But just to get closure regarding my original question, if I still wanted to average correlations, I would simply calculate the mean across all z-transformed r's from one study or sample, right?
Best,
Sara
PS: Thanks for your offer, Jan. I will e-mail you at some point in the future.
The within-study part of the multi-level model takes care of this: you enter several effect sizes for study 1 and several effect sizes for study 2 etc and to synthesize the effect size the model will calculate a weighted average (not a simple mean) and its precision. Depending on the software you use, you will in most cases enter r coefficients (the software takes takes of the transformation).
But if you for some reason want to average r coefficients you need to first transform them into Fischer’s z, average these, and then transform the average back to Pearson’s r.
Thanks.
I have done some reading on multi-level modeling and found a great tutorial here (from the metafor package in r): http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011
Best,
Sara
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis