Hi,
I am providing eta2 for all my ANOVAs. Should I also provide Cohen's d for the post hocs? There are two positions in this area. First, YES, because it does provide some additional information. Second, NO, because it does not correct for multiple comparisons. I am almost convinced for option #2, but it may be, that I should change my mind :-). Please give some advice, with reference at best, since this will help me in the discussion with the reviewer #2 (who must be stopped btw!).
Thanks in advance!
Wojtek
I wouldn't use standardized effect size measures, as they mix signal and noise. For ANOVA, the estimated mean differences are a more useful measure, to my opinion. If you agree, Tukey's HSD provides confidence intervals that are adjusted for multiple testing.
NB: the mean differences and CIs are not really estimates. They are large-sample approximations of estimates, what might be inappropriate for smaller samples and in the precence of prior knowledge. If you take it serious and really want to make a statement about what you can know about the mean difference, given the data, you will have to use a Bayesian approach, and this allows you to consider the multiplicity problem explicitely. See e.g. www.mpe.mpg.de/~aws/Berger.pdf
Hi Wojciech Pisula,
Please note that you should not be concerned with correcting effect size measures in multiple comparisons. Indeed, corrections only apply to probabilities associated with the differences between effect sizes (p values) and not to the effect sizes themselves.
Hello!
many thanks to Jochen Wilhelm and Eduardo R. Cunha for hints!
Wojtek
Since I struggle with the exact same problem, i.e having used partial eta2 for effect size estimates for ANOVAs, but nothing for my 3 group (posthoc) pair-wise comparisons, I'm also most interested in alternative approaches.
However, wouldn't there be a problem "just" using the difference of the means, in the assessment whether it's a large, medium, or small effect (since there is no standardized ES scale for differences of the mean, or is there one?)?
And thereafter, how to compare effect size estimates as such between studies....let's say one study that uses qPCR and another study using WB? Let's say that I want to see if there is a large effect on both protein and RNA transcripts levels from the same gene - could I still use differences between means, and compare them, as a sole estimate of effect size?
Cheers,
Gaetan Philippot, what else would you use instead?
PS: I would not compare the point estimates (like mean differences) but interval estimates (like confidence intervals). Ideally, I would compare posterior distributions (if using confidence intervals, they are interpreted as a crude estimate of the highest posterior density interval (HPD)).
- Badges
- Science topic
- Similar topics
- Correction