Dear all,
Is there a way to convert partial eta^2 to cohen's d for repeated measures designs? I ask because I am conducting a meta-analysis and need to convert the studies' results into a common effect size.
I read that it can be calculated in 2 steps: first cohen's f and then cohen's d.
(1) cohen's f can be calculated from partial eta^2 as follows:
cohen's f = sqrt(partialeta^2/1-partialeta^2)
(2) cohen's f can be converted to cohen's d as follows:
cohen's d = f*2
When I try this with an example from a paper in which there was a partial eta^2 of .42, I get the following:
f=.85
d=1.7
However, I also have the pre and post means and SDs, and when I calculate the cohen's drm according to Lakens (2013), I get very different values. (Note that I do not have the r-value so I will assume one of .5). Mpre = .92, SDpre = .09, Mpost = .98, SDpost = .02.
Cohen's drm = (Mdiff/sqrt(SDpre^2 + SDpost^2 - 2 * r * SDpre * SDpost)) * sqrt(2(1-r))
drm = .73.
This is a huge difference. Even when changing the r-values, this effect size does not get close to the d-value of 1.7 estimated from the partial eta^2 value.
As such, I presume that the calculation I used from partial eta^2 to cohen's d is incorrect for a repeated measures design, and that a correction needs to be applied at step 1 or 2 (or both). I have not been able to find information about this so far. Does anyone know how to do this conversion for repeated measures designs properly?
Thank you in advance for your help.
Hello Caitlin Duncan. I assume this is the Lakens (2013) resource you are referring to, right?
Article Lakens D. Calculating and reporting effect sizes to facilita...
If so, have you considered contacting Daniël Lakens to ask if he is able to help? He would have the advantage over the rest of us of not having to familiarize himself with that resource. ;-) He does have a RG profile, and you can also find an email address for him on the Eindhoven University of Technology website (https://www.tue.nl/en/). HTH.
There may be two problems:
1) First of all, there seem to be two "conventions" how to calculate f^2, as Lakens writes himself and quoted Erdfelder (from G*Power), that the SPSS f^2 and the G*Power f^2 are not the same and have to be transformed into each other. Please see the appendix of the Lakens article- Since I am missing needed information, I could just estimate it, with some reasonable values:
2*sqrt((0.85^2)*((N-k)/N)*((m-1)/m)*(1-r))=0.8284775
where N=20, with k=2 groups, m=2 repeated measures and r=0.5, which seems more reasonable (the 2*sqrt is already the transformation into d). But I do not know where you got partial eta from, therefore, this is just a guess.
2) Another point may be the use of partial eta squared itself, since it partials out the variance of other factors in the model (also from the DV), so that it may be inflated as compared to the effect itself measured with the raw values.
In either case, I would use the raw values and consider using Cohens dav or Hedges gav instead of drm.
Rainer Duesing Thank you for pointing out the equations in the appendix section.
In the particular example I gave (N=26, sorry I didn't include that in the original post), the researchers used R to compute the ANOVA and the partial eta^2.
To continue with this example:
cohen's dav = 1.09; hedges gav = 1.06.
cohen's drm = .73; hedges' grm = .71
converting from partial eta^2, as per your recommendation, cohen's d = .85, hedges g = .82.
Calculating the cohen's f and cohen's d the way you did yields a more reasonable result, e.g. .85 instead of the 1.7 that I calculated in the original post. It would seem then that cohen's f is calculated in R as it is in SPSS.
As you recommend, I will definitely use the raw values when available. However, in some articles, the partial eta^2 is the only value given, hence my search for a proper conversion.
With respect to Cohen's dav/Hedges gav, I agree that for most cases those values make sense - in particular for combining effect sizes from different experimental designs. However, as I am performing a meta-analysis only on pre-post within-subjects design (e.g. single group), my understanding was that Cohen's drm/Hedges grm would be appropriate because it takes into account the correlated nature of the data. Is your understanding that it's more appropriate to use Cohen's dav/Hedges gav in this case? Is cohen's drm/hedges grm simply too conservative?
Thanks!
I am not sure about your calculations for drm/grm, my results show .92 and .89, respectively,
There is an ongoing debate, which effect size to choose in case of repeated measures. Here are some thoughts about it, which I wrote for myself in preparation of a paper (maybe published sometime, hehe), the notation is taken from the Lakens article:
"Under the assumption of equal variances, dz is larger than the estimate for independent observations and hence is standardized by the factor to account for it, which gives drm. Under equal variances and a correlation of r = .5 dz and drm will have identical results and also in comparison to ds for independent samples and dav (see next section), but they will deviate with increasing heteroscedasticity or when r approaches 0 or 1 (Lakens, 2013). Especially dz tends to overestimate the ES in case of high correlations (Kline, 2013) and can give highly misleading results (Bonett, 2015), although it can also be argued that the same difference between groups/conditions can be better detected in dependent designs, which is in turn reflected in larger dz values (Baguley, 2012). Additionally, dz and drm are in the metric of difference scores and not the original units of the variables, which can be more difficult to interpret, especially under the aforementioned conditions. For research synthesis and meta-analysis drm may also be problematic, since the intercorrelations of variables are not always entirely reported in original research articles, so that drm cannot be calculated or r has to be set to a plausible value. There seems to be agreement typically not to use dz for calculation of within-sample ES and drm only if this metric is specifically requested, e.g. when the emphasis is on measurement of change (Cumming, 2013; Cumming & Calin-Jageman, 2016; Cumming & Finch, 2001; Kline, 2013).
The second approach is to calculate dav, which uses the pooled (averaged) standard deviation of both conditions as denominator. The advantages of dav are that it remains in the original metric of the variables, it is identical to the according ds measure for independent samples (when samples sizes are equal), and will generally be more similar to ds than drm (Lakens, 2013)."
Therefore, my argument would be the other way around, since you do not know or assess heteroscedasticity explicitly and in many cases do not know the correlation (as in your case where you have to make a wild guess), drm/grm gives very varying and maybe misleasing results. Just test it with changing values for r. Additionally, dav is often closer to the between subjects ds, and even identical with equal sample size, and therefore more comparable if for example the same experiment had been conducted as within and as between subject design. But I think there is no definitve answer to the question which ist "better". If you are completely unsure, maybe use both and compare the results and report both if they diverge extremely. This could also be a vluable information for further research.
Rainer Duesing Thank you for this thoughtful response. Using dav/gav would indeed be more straightforward for a meta-analysis for the reason that the r value is very rarely reported and therefore has to be requested from the researchers for each article, or imputed, and of course imputation could lead to erroneous estimates as you mention.
I am still curious about how one can convert from a partial eta^2 value to a cohen's d/hedges g value for repeated designs without having the r-value. The equations that you pointed out in Lakens' (2013) appendix show that to get the "proper" G*power version of the f^2 value, one needs the correlation between the two measures.
One idea is to convert the partial eta^2 into generalized eta^2, and then using the d =2* sqrt(etaG^2/1-etaG^2). However, calculating generalized eta^2 for repeated designs requires that one has access to the full ANOVA table, which again, is not often reported (also discussed in Bakeman, 2005, link:Article Recommended effect size statistics for repeated measures designs
) It's worth noting that partial eta2 and generalized eta2 are equivalent for a one-way between-subjects ANOVA.Alas, it seems I'll have to contact the authors either way - requesting the r value or simply a different effect size.
Cheers!
Hello Caitlin
I have the same need for my meta-analysis - authors have provided a η2 effect size, and i need to convert it to Cohens d. Can you please let me know how you resolved this? Also, I wonder where you found the formula
cohen's f = sqrt(partialeta^2/1-partialeta^2)
as i will need to cite an authority!
Thanks in advance.
SPSS cannot calculate Cohen's f or d directly, but they may be obtained from partial Eta-squared. Cohen discusses the relationship between partial eta-squared and Cohen's f :
eta^2 = f^2 / ( 1 + f^2 )
f^2 = eta^2 / ( 1 - eta^2 )
where f^2 is the square of the effect size, and eta^2 is the partial eta-squared calculated by SPSS. (cf. [Cohen], pg. 281.) Therefore,
f = sqr( eta^2 / ( 1 - eta^2 ) ).
Cohen's f = Square Root of eta-squared / (1-eta-squared) If the model is a Univariate ANOVA with two groups, and the number of observations in each group is equal, then the standardized range of population means, Cohen's d, is given by
d = 2*f
Cohen's ds is sometimes referred to as the uncorrected effect size. The corrected effect size, or Hedges's g (which is unbiased, see Cumming, 2012), is:
Hedges's gs=Cohen's d/(1−(3/(4(n1+n2)-9)))
- Badges
- Science topic
- Similar topics
- Psychology