I am working with count data (count of organisms surviving after different drugs), best fitted by the negative binomial distribution. I ran GLMM on the data and then used emmeans to estimate marginal means and perform pairwise comparisons to find the best-performing drugs. I now want to calculate standardized effect sizes to measure the effect of significantly better-performing drugs compared to control. However, I am skeptical about using Cohen's D, as my data is non-normal. My question is, what else can I use in this case? Or, perhaps, if I am already dealing with marginal means (estimated on the model, not the data) and have compared them with Tukey, am I safe to use Cohen's d measure?
What software are you using ? In R, the emmeans package has a standardized effect size function--- equivalent to Cohen's d in simple cases --- based on the emmeans analysis.
Sal Mangiafico Thank you for your response. I am using the emmeans package and am aware of its eff_size function. My question is more about whether it would be right to use that function, as it utilizes standard deviation and assumes normality (in my knowledge), while my data is non-normal count data.
Hello Susanna Avagyan Maybe the link below can help you, especially section 2.5
https://transparentstats.github.io/guidelines/effectsize.html
It's an interesting question. Basically, you want the difference in e.m. means divided by an appropriate measure of pooled variance. This would be similar in spirit to Cohen's d, and be what people think of a standardized effect size. This is obviously what the eff_size() function aims to do. The catch is determining what the appropriate measure of pooled variance would be for various glm models. The documentation for eff_size() gives some examples for how to specify sigma and edf, but I certainly don't understand it well enough to know what would be appropriate for various glm models. For OLS models, the following appears to work, eff_size(marginal,sigma = sigma(model), edf = df.residual(model)) , where marginal is the emmeans object. But I'm not sure about for other types of models.
Perhaps the most fruitful approach would be to write to the package author, and ask for your specific model what the appropriate values for sigma and edf would be.
You might also try posting on Stack Overflow, as the package author, Russ Lenth, does respond to questions there.
See also: https://stackoverflow.com/search?tab=newest&q=eff_size
I guess the first question is why you need standardised effect size at all? Generally for interpretation the unstandardised effect size - differences in the counts or rates are more useful. Similarly for comparing between models the counts or possibly log(counts) are likely more useful.
In a standard regression model standardised effect size is useful for power analysis, but that's not the case for a negative binomial model where you'd probably want to use the log(counts) and estimates of dispersion etc.
In standardised effect size you'll be scaling the effects according to an estimate of noise in the model and if you have over dispersion I'd bet your estimate of noise is going to be fairly unstable (e.g., if comparing different groups or between studies).
Its possible you might want to do this for a meta-analysis (though even then it might be better to use a different approach).
In summary, it depends why you want Cohen's d.
Besides Cohen d, there are many other effect size estimators( more than 100 :) ). Cliff's delta does not depend on the type of distribution, it is suitable for measurements on a scale not lower than ordinal.
it depends why you want Cohen's d. (© - Thom Baguley )
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis