I am using Lavaan and semTools for testing the measurement invariance of a test in two large cohorts (n1 = 1520; n2 = 1178). There are no missing data and the tests should have a high power. The downside: Already small disparities get significant. Is there a correction for the Chi Square test on testing the modell fit of the restricted modells or different approaches that are less susceptible to the sample size of the test data?
Dear Wolfgang,
Try the guidelines for RMSEA (Chen, 2007) and for CFI – differences in CFI less than .001 (Cheung & Rensvold, 2002) or less than .002 (Meadeet al., 2008). According to Chen (2007), for a cut-off point of .05, only a value equal to or less than -.005 in the CFI associated with a value equal to or greater than .010 in the RMSEA indicates non-invariance .
Chen, F.E. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, pp.464–504.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568-592.
Good luck with your research!
Dear Wolfgang,
Try the guidelines for RMSEA (Chen, 2007) and for CFI – differences in CFI less than .001 (Cheung & Rensvold, 2002) or less than .002 (Meadeet al., 2008). According to Chen (2007), for a cut-off point of .05, only a value equal to or less than -.005 in the CFI associated with a value equal to or greater than .010 in the RMSEA indicates non-invariance .
Chen, F.E. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, pp.464–504.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568-592.
Good luck with your research!
In addition to Lisete's recommendations, there is another more recent assessment:
Fan, X., & Sivo, S. A. (2009). Using Δgoodness-of-fit indexes in assessing mean structure invariance. Structural Equation Modeling, 16(1), 54-69.
You might also want to have a look into the attached book. The authors propose a method to compute the appropriate sample size for testing measurement invariance. It comes with the according R package OPDOE:
Simecek, P., Pilz, J., Wang, M., & Gebhardt, A. (2014). OPDOE: OPtimal Design Of Experiments. Retrieved from http://CRAN.R-project.org/package=OPDOE
Accordingly, in this case, it might be an appropriate approach to select a subsample of the required size using stratified randomization.
----------------
Rasch, D., Kubinger, K., & Yanagida, T. (2011). Statistics in psychology using R and SPSS. John Wiley & Sons.
Kubinger, K. D., Rasch, D., & Yanagida, T. (2011). Statistik in der Psychologie: vom Einführungskurs bis zur Dissertation. Hogrefe Verlag.
Book Statistics in Psychology using R and SPSS
Book Statistik in der Psychologie – Vom Einführungskurs bis zur D...
All the relevant references have been listed already. One correction to a comment above, maybe a typo: According to Cheung & Rensvold (2002, p.251) Delta(CFI) > .01 between consecutive models in invariance testing is indicating a substantial deterioriation in model fit.
BTW: The measurementInvariance() already provides all the necessary comparisons: http://www.inside-r.org/packages/cran/semTools/docs/measurementInvariance
Kind regards, Ulrich
http://www.inside-r.org/packages/cran/semTools/docs/measurementInvariance
Sorry for the late post, but another option would be the test of small difference in fit by MacCallum et al. (2006) to test nested models with large sample sizes
MacCallum, R. C., Browne, M. W., & Cai, L. (2006). Testing differences between nested covariance structure models: Power analysis and null hypotheses. Psychological methods, 11(1), 19-35.
Here you find R and SPSS codes:
http://timo.gnambs.at/en/scripts/testofsmalldifference
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size