I am currently analyzing data from a behavioral study on emotion perception using lmer model comparisons in R. I have a fairly simple design, with emotional intensity as repeated measure and valence (positive, negative) and group (controls, patients) as fixed effects. Random effects are subject (n=37) and stimulus (n=8). I am comparing a simple model with more complex models using the anova function, but I ran into two problems. My degress of freedom for one of the model comparisons is 0, and it seems to matter in which order I put the models when comparing them. Here's the set up and outputs:
Models:
intensity.valence.1
Your model with just the fixed effect of valence and your model with just the main effect of group are not nested. Each these should be compared with the first model and the one with both.
Your model with just the fixed effect of valence and your model with just the main effect of group are not nested. Each these should be compared with the first model and the one with both.
As far as I can tell, the anova function does this odd thing where anova(model.1, model.2, model.3) compares model.1 to model.2, then model.2 to model.3. *
Similar functions in other packages have you explicitly select one model to compare all other models to. Which makes more sense to me.
I think the best practice is use anova with two models at a time. Even though it's a little more tedious, the user is less likely to misinterpret results.
______________________
* Update: Stephen Politzer-Ahles notes below that for lmerMod objects, the anova function tries to identify the smallest model to compare to. This seems to give more support for the idea that the safest practice is to compare models two at a time to avoid any confusion.
Salvatore is correct that the anova for class merMod uses the same rule as the base stats anova. It tests them in sequence. As well as requiring a work around if you don't want that, also it means that people have to be careful that the models are on the same dataset and do not have different amounts excluded for missing values. If your exclude data if there is a missing value for your 8th model, you need to exclude for others with which you compare it. How you deal with model selection with these models can be tricky.
In my experience, if you test models that where one is a subset of another (e.g., m1
you should test only two models in one anova.
e.g. start with the null model vs. the simplest model and then compare the simplest model to the next more complicated model. null vs. main effects, main effects vs. interaction and so on.
Hi Kathrin!
I suggest you to read these papers..maybe it can help!
Get a look to this also..
http://stats.stackexchange.com/questions/83778/likelihood-ratio-test-lmer-r-non-nested-models
Article A.F. Zuur, E.N. Ieno, N.J. Walker, A.A. Saveliev and G.M. Sm...
Article Mixed-Effects Models in S and S-Plus by Jose C. Pinheiro; Do...
Sorry in advance if this is out of subject.
@ Katerina Danae Kandylaki, I am currently doing this approach for my study, and I was wondering whether we had to control the threshold of the p-value of each comparison for multiple testing, e.g. using a Bonferroni correction? Thanks !
Hi Kat!
this question is old but I've just been going back to this stuff.
Possible anti-conservative issues aside and once you have the correct nesting of models, I don't see the problem with chucking in all your models into the anova function at once. Since the function is going to compare the most complex model to the next-most complex model automatically, I'm not sure why one would need to do it two models at a time (assuming the random fx stay the same)? Or am I missing something?
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA