Dear community,
say one wants to report a linear mixed model of a psychological experiment (mixed design: predictor A between, three levels; predictor B within, four levels).
What would be the best way to report it?
The F test is not that informative when the predictors (fixed factors) have more than two levels (see Schad et al., 2020)
The summary() command only provides the comparison to the reference category (however, lets say we are interested in more than these to-reference-category-comparisons)
The usual repeated-measures-ANOVA-type-of-reporting would be F test and post-hoc tests. However, would you say this applies also to LMM?
I would also argue that it makes sense to report the model comparisons (LRT) of the model that contains the set of fixed effects of the interaction (the “interaction model”) with the main effects only model. And before to evaluate the main effect of predictor A by comparing it to the intercept-only model.
so take the model comparison lmm1
J. Pallant (2016) SPSS survival manual, pages 167 -168 gives you a good example how to present your results.
It sounds like you have a good approach to how to handle this analysis...
1) Overall model. Yes, you can present the overall significance test, such as the LRT test. This would be conducted in comparison to an intercept only model. Or skip this if you're not interested.
2) Overall fixed effects and random effects. You can also compare your model to a random-effects-only model or a fixed-effects-only model to tease out the effects of fixed and random effects. Or skip this, and just rely on 3).
3) Tests for factors. If you are using lmer to fit your model, the lmerTest package is useful to get test the individual fixed factors and individual random factors. (Like in a common anova.) See the anova.lmerModLmerTest and ranova functions. Using these functions, you don't need to conduct these tests manually.
4) Post-hoc analyses. The emmeans package should be appropriate for conducting post-hoc tests for significant effects or interactions. Typically, all pairwise comparisons of E.M. means for main effects, interactions, or one effect within another, an so on. The lmer example here is somewhat outdated, but mostly works ( https://rcompanion.org/handbook/I_07.html ), with the caveat that I wrote that I wrote that page. The lsmeans package can be updated to the emmeans package, and the CLD part doesn't work anymore, but there's a way to get this if that's what you want.
5) The emmeans package is also useful for conducting specific custom contrasts. ( See, e.g. https://rcompanion.org/rcompanion/h_01.html , the sections with the lsmeans package, with the caveat that I wrote this page).
6) You can report results of custom contrasts or pairwise comparisons, or both, depending on what is of interest.
In my experience telling what you did followed by what you found that was important is a good strategy..But hey I'm trained as a P Chemist. Attached is how.we do the report in our world. Best wishes, David Booth
- Badges
- Science topic
- Similar topics
- Statistics
- Time Series