21 December 2022 3 899 Report

When conducting post hoc tests for mixed models (lme4 package), the most commonly cited method is to use the package "emmeans" which conducts a contrast analysis. I have slowly shifted towards using this but have some reservations that I would feel more comfortable with some clarity on.

When using emmeans, the Standard Errors (SE) are the same for each contrast, which is because emmeans is taking into account the assumption of homogenous error variances and equal numbers of observations between groups (homoscedasticity). However, mixed models, unlike traditional linear regressions, can account for missing observations and different variances between groups with random effects. Thus, the output for the "emmeans" showing equal SE's for each contrast raises some questions for me.

When I use multiple filtered mixed models for post hocs instead, I get the same "Estimate" (estimated difference between groups), but obviously the standard errors are now not homogenous across contrasts which slightly affects p-values. I will note that this method does not drastically change results as differences in SE are around the value of the homogenous SE from emmeans output.

In short, my question is whether emmeans should not assume equal variances for post hoc tests of mixed models because of random effects.... and if this is the case, is using individual models preferrable to remain consistent with original model parameters/assumptions? Perhaps I may be missing something fundamental or obvious. Any input would be helpful! Thanks!

More Brady DeCouto's questions See All
Similar questions and discussions