I am analyzing results from a daily diary study in R using the nlme package. I am using hierarchical linear modelling and am not sure which code to use to examine whether the latter model fits better than the previous model.
I tried the code anova (model1,model2) but it is giving me the following error message: Error in anova.lme(Model1.SigControls.IIP, Null.Model.IIP) :
all fitted objects must use the same number of observations
In addition: Warning message:
In anova.lme(Model1.SigControls.IIP, Null.Model.IIP) :
fitted objects with different fixed effects. REML comparisons are not meaningful.
I am not a R user but this should help.
If the models differ in the their fixed part the Likelihood Ratio test should be done following FIML estimation
If they differ in the random part then the LRT should be based on REML.
REML is like useing n-1 when calculating a variance as 1 degree of freedom has been used by calculating the mean; FIML does not take account when estimating the random part that the fixed part is an estimate.(Full Bayes analysis has complete uncertainty propogation - both fixed and randomn paraeters are seen as estimates when estimating the other part.)
It is also often suggested with REML that when tesing models that are different in their random part the p should be divided by 2 to get towards a 1 tail test
See the folowing which has a chapter on testing (with references) albeit for MLwin
https://www.researchgate.net/publication/260771330_Developing_multilevel_models_for_analysing_contextuality_heterogeneity_and_change_using_MLwiN_Volume_1_%28updated_June_2014%29?ev=prf_pub
Hope this helps but I am sure a R user will give you chapter and verse.
Book Developing multilevel models for analysing contextuality, he...
Alex,
What are your two models and the ns for each model? I do mostly use R so can see if I can replicate the error and help.
From the message it looks like there may be missing values for one model but not for the other. For example, if you add a covariate that has missing values and so the procedure drops say 5 cases, the two models will have different ns so you need to drop those cases from the previous model.
Dan
As Kelvin noted, you need to compare models that differ only in the fixed part using ML estimation rather than REML. One solution is to refit the models with lme4. The anova command there uses profile likelihoods (rather than REML) to compare models and this avoids the problem with using REML fits to compare fixed effects.
The other error message refers to having different n (presumably because of missingness on one or more predictors). This is a common issue with use of any likelihood based statistic. The simple solution is to run the models on exactly the same data (e.g., through some kind of imputation approach or - less satisfactorily - by omitting cases).
At Thom said, you could re-run your models using the 'lme4' package in R, and include REML = F in each model, e.g.
model1
The default for the anova function applied to an object from lmer (of class lmerMod) is that if REML was used to refit without reml. You can control this but for reasons given by Kelvyn and Thom this is the default.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Software
- R