Hello,
I'm running a one-way ANCOVA to compare a ratio variable between two groups and adjust for some confounding variables. The software I use (SPSS) reports partial eta squared as effect size statistic. However, I would like to know if it's valid to calculate Cohen's d for this analysis.
I first thought to use the least squares means, standard error of the mean (SEM) and n reported for the estimated marginal means from the ANCOVA analysis. First, multiply SEM * SQRT(n) to get each group's standard deviation (SD) and then calculate the pooled SD to calculate Cohen's d.
However, I also found that Fritz et al. reported an equation to calculate Cohen's d from eta squared as d = 2 * SQRT(eta squared) / SQRT(1 - eta squared).
Would Cohen's d calculated from the procedures above be valid for ANCOVA? Or should I report eta/partial eta squared?
Thank you in advance.
Alejandro.
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141(1), 2–18. https://doi.org/10.1037/a0024338
I would be cautious about using those types of equations for converting among effect size statistics. They often have assumptions or limitations wrapped up in them.
What software are you using ? There may be a specific method for the lsmeans and a Cohen's d - like statistic.
It probably does make sense to report eta-squared or partial eta-squared.
What is the dependent variable? If the DV is familiar to and well understood by your readers, why not simply report the (adjusted) mean difference between the groups? See this article by Thom Baguley for relevant discussion.
https://pubmed.ncbi.nlm.nih.gov/19017432/
Article Standardized or simple effect size: What should be reported?
HTH.
When conducting an ANCOVA, you take into account one or more covariates in your analysis, which makes it more complex than a simple two-group comparison. In this situation, Cohen's d can still be used to measure the effect size, but it may not be the most suitable measure. Other effect size measures, such as partial eta squared or omega squared, may be more appropriate for ANCOVA as they consider the number of predictors and degrees of freedom in the model.
if you don't know softwares like R, then you can use websites that do the effect size conversion for you like Psychometrica (https://www.psychometrica.de/effect_size.html).
Taking a step back the answer is that it depends what you are trying to do. In a strict sense Cohen's d isn't defined for ANCOVA only for a two independent group design and therefore there's several plausible and less plausible ways to compute a standardised mean difference (smd) depending on the details of the study (homogeneity of variance, choice of covariates and so on).
As Bruce Weaver mentioned I think that the adjusted mean difference (unstandardised) is arguably a superior effect size statistic for interpretation.
If you are interested in power analysis for a future replication with the same design etc. then computing the smd then using th adjusted mean difference and the root mean square error from the analysis makes some sense. This will reduce the estimate of variability and capture the increase in power typical from including covariates.
However, this is not sensible for interpretation. Just because you have a more precise estimate doesn't mean the effect actually got bigger. Also it will be scaled differently from a two independent group design (a little like comparing two effects scaled as cm and inches or seconds and minutes). For the smd in this case I'd scale the adjusted mean in terms of the raw SD of the outcome (Y) variable).
Hello everyone.
Thank you for all of your answers. I will definitely go for eta or omega squared to report effect sizes for ANCOVA. I thought it would be odd to report eta or omega squared for a two-group comparison instead of Cohen’s d. It just made sense to me to estimate Cohen’s d using the least squares means and the standard error of the mean (by calculating the corresponding standard deviations and a pooled SD) because both are modified by the confounding variables in the model.
Thank you Bruce Weaver and Thom Baguley for the references and clear explanation. Using the non-standardized difference is something that I was planning to report; however, I also wanted to use a standardized difference effect size to account for variability.
Best wishes,
Alejandro.
Alejandro Gaytán González I'm not sure what you mean when you write "I also wanted to use a standardized difference effect size to account for variability." In my opinion it doesn't account for variability at all ... rather it scales the estimate of the effect in terms of SD or variance. You account for variability (again in my view) by using an interval estimate rather than a point estimate.
It's possible you just meant scaling in terms of variability. Scaling in terms of SD or variance can be useful in some situations (notably for power and sample size calculations for a study with a similar design and population). However, for interpreting an effect its a problem because the "size" of the effect depends on several different sources of variability that may not be relevant to your problem and also depends on how the design treats those sources (individual differences, sampling error, measurement error, systematic differences in variability between samples/populations, stimulus/item variability and so on). I think its cleaner (in most cases) to separate out the effect size estimate from its variability.
- Badges
- Science topic
- Similar topics
- Geoscience
- Asia
- India