Dear Experts,
I need your guidance on the invariance test performed via AMOS. My friend is comparing two groups, A and B. Their Model fits are excellent, less than 3. CFI is 0.98 for both groups. RMSEA group A is 0.040, and group B is 0.038. All factor loadings are above 0.8 for both groups, but there is a huge chi-square difference of about 400+ between constrained and unconstrained models. The sample size is large 1600+. I need your guidance to resolve this issue in order to establish the invariance between the two groups. Your expert opinion will be highly appreciated. Thank you so much
With this large sample (> 1,600 cases), you have a lot of statistical power to detect differences in fit between nested models via the chi-square difference test. This is, first of all, a good thing. Even minor misfit (= small departures from measurement invariance/equivalence) can be detected.
To find out whether the large chi-square difference corresponds to substantively/practically meaningful differences in, for example, loadings or intercepts between groups, I would inspect modification indices. These may show you whether any differences in loadings etc. are sizeable and relevant for the interpretation or not. I would start with a comparison of a model that only constrains the loadings to be equal across groups (weak or metric invariance) to the configural invariance model. If the weak invariance model already fits worse than the configural model, you'd have to figure out first which loadings are non-invariant (whether this matters) before moving on to intercept invariance testing.
Dear Professor Christian Geiser, Thanks for your insightful feedback. Both groups compared which have good fits, all fit indices like NFI, TLI, and CFI are >.095, and loading values are also > 0.80. Due to the large difference (400+) in chi-square values, the chi-square difference test is not invariant. What further measures need to be taken to rectify this issue? Kindly guide. Thank you so much
Dear Asif,
Before testing measurement invariance, you need to check and control for Common Variance Method Bias (using Random-Intercept CFA model). To do so, you may use the procedure that Steenkamp and Maydeu-Olivares (2021) have proposed. Next, you have several options depending on whether CMV is an issue or is not. Anyway, when the gap across unconstrained and constrained model is significant, you may relax some constrains (based on modification indices) on loadings, then on intercepts to achieve at least partial invariance (see Byrne et al, 1989). Keep in mind that partial metric (scalar) invariance is needed to compare paths (means) across groups.
Hope this helps,
Best, Eric
Byrne, Barbara M., Richard J. Shavelson, and Bengt Muthén (1989), ‘'Testing for the Equivalence of Factor Covariance and Mean Structures: The Issue of Partial Measurement Invariance,” Psychological Bulletin, 105 (3), 456–466.
Steenkamp, Jan-Benedict E.M. and Alberto Maydeu-Olivares (2021), “An Updated Paradigm for Evaluating Measurement Invariance Incorporating Common Method Variance and its Assessment,” Journal of the Academy of Marketing Science, 49, 5–29.
See here https://stats4edu.com/services/structural-equation-modeling/
- Badges
- Science topic
- Similar topics
- Correction