Cohen's d can be calculated from ANCOVA, using the following formula (presented in Borenstein M. Effect sizes for continuous data. In: The Handbook of Research Synthesis and Meta-Analysis. 2nd ed. 2009):
d = ( F × (n1+n2) / (n1×n2) ) × SQRT(1 - R2)
What is the R in this formula? Is it bivariate correlation? Or partial correlation (partial eta squared)? Or even part correlation (classic eta squared; i.e. proportion of variance explaned by the covariate)?
I reviewed ANCOVA formula to see whether could I find which one; which didn't help with the asnwer, but may explain the question better:
The ebove mentioned formula has two parts: the first is the same formula for dalculating d from ANOVA (without any covariate). The second part seems to be the part that we count for the adjustment in ANCOVA. In ANCOVA when we did the adjustments, we increased the F, and now it seems that in this formula we want to count for it.
In ANCOVA (with two groups), we use two correlations to adjust for SSs:
First we use bivariate (zero-order) correlation to adjust for SStotal (i.e. SSt[adj] = SSt[un-adj] × (1 - R2));
And a partial correlation (partialling out variations due to grouping factor) to adjust for SSresid (i.e. SSresid[adj] = SSresid[un-adj] × (1 - R2)).
Therfore, when we calculate the F, the denominator is adjusted by bivariate correlation and in the numerator by both partial and bivariate correlations, as MSeffect is MStotal (adjusted by bivariate correlation) minus MSerror (adjusted by partial correlation).
So, which kind of correlation should be used in the formula for calculating cohen's d?
Thanks for your time.
(AN UPDATE: I think I found the answer)
As I'm not an statistician (unfortunately) and I couldn't figure out how the formula works, I generated a simulated dataset using "rnorm()" function in R. As ANCOVA is basically about increasing precision, I assumed that in a simulated dataset, the cohen's d would be aproximately the same whether it calculated based on the usual formula (i.e. (M1-M2)/SDpooled) or from ANCOVA. So, I calculated d onse from means (usual formula) and again from ANCOVA with the three types of correlation.
I mentioned the R codes I used here, which seems to show that the R is the partial correlation:
#generating a dataset with pre-post-test design and a grouping factor, and with a 1 score mean difference between groups
pre
Hello Ali,
For equation (12.24) in Borenstein's chapter, he explains R-squared as the "...covariate outcome correlation..." which I take to be the (square of the) correlation between scores on the covariate and scores on the dependent variable.
Good luck with your work.
Hello David,
Thanks for your reply.
I think I didn't present the question properly. As you mentioned R in the covariate outcome correlation. The problem is that the covariate outcome correlation could be bivariate correlation, or partial correlation (the covariate outcome correlation after controlling for the other factor in the model, which is identical to square root of partial eta-squared in ANCOVA), or semi-partial correlation (also called part correlation [it's very similar to partial correlation with a small difference]; which is identical to square root of classic eta-squared).
But, I think it's the partial correlation, as in a simulated large dataset I calculated the d based on these three types of correlation, and the partial correlation lead to a result approximately equal to the one from the basic formula for Cohen's d. (I mentioned the analysis here)
Thanks again.
Note that the t- and F-based formulas for computing d from ANCOVA or multiple regression result given in the Borenstein chapters are not strictly accurate. They ignore the effects of the covariates on the t or F value (and also don't consider the consequence of whether the authors computed t or F using Type-2 or Type-3 sums of squares).
The correct formula for t would be
d= t × SEβ ÷ √(MSE) × √(1 − R2−j)
Where t is the t test statistic for the adjusted mean difference between the groups, SEβ is the standard error for the mean difference (if not reported, SEβ = t × [Adjusted mean difference, i.e. β]), √(MSE) is the square root of the mean square error (or σ from a regression model) and R2−j is the R2 (or η2) for a model predicting the dependent variable using all the other predictors except this group contrast.
The correct formula for F would be
d= √(F) × SEβ ÷ √(MSE) × √(1 − R2−j)
Where √(F) is the square root of the F statistic for the contrast and the other terms are as defined above. SEβ is often not reported in an ANOVA table, so it can be computed using SEβ = √(F) × [Adjusted mean difference, i.e. β].
If you use the formulas given in the chapters, they will tend to overestimate the d value (though they may underestimate if there is high collinearity among the predictors).
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis