Hello everyone!
I currently have 2 measurement models. All are correlated factor models and the factors reflect subscales of anxiety related constructs. Estimator is robust maximum likelihood (to account for lack of multivariate normal distribution). This happens in the context of construct independence. First model implies independence between all subscales. Second model clusters factor 1 and factor 2 together, and factor 4 and factor 5 together.
model 1:
f1 =~ item 1 + item 2 + item 3
f2 =~ item 4 + item 5 + item 6
f3 =~ item 7 + item 8 + item 9
f4 =~ item 10 + item11 +item12
f5 =~ item 13 + item14 + item15
f6 =~ item 16 + item 17 + item18
f7 =~ item 19 + item 20 + item21
model 2:
f1,2 =~ item1 + item2 + item3 + item4 + item5 +item6
f3 =~ item 7 + item 8 + item9
f4,5=~ item 10 + item11 + item12 +item13 + item 14 +item15
f6 =~ item 16 + item 17 + item18
f7 =~ item 19 + item 20 + item21
Nested models are models wherein all parameters of a more restricted model are included within a less restrictive one. I am new to this, and I reviewed examples, but I cannot make a conclusion. I would appreciate an answer and the reasoning behind it a lot!
(note: based on the fit indices, I know that the second model does not work. Fit indices are well below the acceptable threshold and the differences are enormous. But in the case of non-nested models, comparison of absolute and relative fit indices is not the case and I want to do the comparison anyways to learn how to do it correctly)
The models are nested. You can see this from the fact that an equivalent way to specify Model 2 is by setting two factor correlations in Model 1 to 1.0, namely between Factors 1 and 2 as well as 4 and 5.
There are two complications in this case, however, when it comes to applying nested model chi-square difference testing. First, correlation coefficients of 1.0 are at the boundary of the admissible parameter space for correlations, violating assumptions of the chi-square test. Stoel et. al. (Psychological Methods) developed a correction for this. The second complication is the non-normality correction of the chi-square values for each model. You would have to take the scaling correction factor into account when computing chi-square difference tests. It is perhaps easier to simply compare the absolute fit of the models or look at information criteria such as BIC.
Christian Geiser
I am working with a fairly large sample so chi-square would be inflated and not so informative. You mention absolute fit indices, in which I can see strong differences between my models. Would you also use differences in CFI and TLI? Cheung and Rensvold (2002) suggested the CFI difference between two nested models (∆CFI > .010) can also be used.
For AIC and BIC comparisons, I see them mostly in comparisons of non-nested models. There is the nonnest2 package in R in which the icci command calculates confidence intervals for the AIC and the BIC but for non nested models. How would you go about calculating CIs for the AIC and BIC in nested models?
Thank you for helping me out again!
If the overall fit statistics clearly favor Model 1, then why not just use that information to choose Model 1 over Model 2?
I don't agree that chi-square is "inflated" by sample size. It is natural for a test of significance to have greater sensitivity (power) with increasing sample size. It is actually very informative with large N because it has a higher sensitivity for detecting model misspecification(s).
The problem with indices like CFI, TLI, etc. is that they loose sensitivity with increasing sample size which is counterintuitive and not helpful in my opinion (see Article In Search of Golden Rules: Comment on Hypothesis-Testing App...
)I don't know about CIs (because I have always considered the AIC and BIC indices to be merely descriptive in nature), but you can use AIC and BIC for both nested and non-nested model comparisons.
Christian Geiser This is what I am gonna do. But I wanted to treat this problem as if this was not the case so that I struggle more with the logic of comparing models and learn it better.
Thank you a lot for sharing the article! I was under the impression that cut-offs for fit indices are well established and generalizable.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size