Does anybody have suggestions on how to evaluate invariance of structural parameters across large (n > 350) groups?
My dilemma is that in the models the delta CFI consistently falls between the .002 and .01 cut offs.
Meade, Johnson and Braddy (2008) suggest the McDonald's NCI as best choice for testing for measurement invariance in large groups (>400) and provide cut off values in Table 12. Are these also applicable for testing invariance of structural parameters?
For example: is the influence of Pessimism (3 items) on Trust (3 items) invariant across 8 groups (n ranges from 350 to 1000)? The difference of the McDonald's Non-centrality Index between model 1) in which only loadings and intercepts are constrained to be equal and model 2) in which additionally the structural path from Pessimism to Trust is constrained to be equal is 0.0026.
I'm also curious whether there is a standard way of evaluating invariance of structural coefficients when n is larger or if you have any suggestions on how to proceed? Delta CFI does not seem to help me here.
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568–592. http://doi.org/10.1037/0021-9010.93.3.568
There is a paper that evaluated about 20 Goodness-of-fit indices (GFI) and found that delta CFI, delta Gamma hat, and delta McDonald's Non-centrality Index are independent of sample size and complexity, and are not correlated to overall fit measures. (The recommendation was reached through a multi-group situation, analyzing the changes in GFI). Please find the paper attached. Hope it helps in your research.
Hello Joachim,
I think you have it almost ready. As far as I know, the difference between the commonly known measurement invariance and relational invariance (path coefficients invariance) is that the later needs the former to be sustained. i.e. Relational invariance is kind of a 'higher' level of invariance. I think cutoffs should be the same; and haven't found sources that says something different.
That makes sense to me since the principle is the same: apply some theoretically relevant constraint; if global fit doesn't changes (in a statistically significant rate), then your model is 'better' than before, i.e. it can be applied in more situations.
Traditional measurement invariance (from configural to mean invariance) does this step by step, but doesn't refers to path coefficients. This sometimes calles relational invariance would be a further constraint over measurement invariance, but shouldn't change the fundamental criteria. Cheung and Resvold (2002) and Mead et al (2008) are respectively the commonground standard and the ideal standard for invariance cutoff criteria. Both papers put CFI and NCI as good indices. It's important to take notes on the differences between both studies to know which cutoff values are the best on your own analysis; we are always between type I and type II inferential errors.
Bottom line: I think you should use one cutoff criteria for every step of invariance testing from configural to realtional. That implies that relational invariance should use the same basic rationale and criteria than measurement invariance.
A recent paper ontopic, where you can find references to specific relational invariance papers:
Guenole N. and Brown A. (2014) 'The consequences of ignoring measurement invariance for path coefficients in structural equation models'. Front. Psychol. 5:980. http://dx.doi.org/10.3389/fpsyg.2014.00980
Cheers!
Dear Mr. Rahman and Mr. Torres Sahli,
thank you for your responses and the articles!
In my experience, chosing the .01 or the .002 cut off often lead to contradicting outcomes.
I just found some discussion about this in
Cigularov, K. P. (2008). Achievement Motivation in Bulgaria and the United States: A Cross-country Comparison. ProQuest.
on page 63 for those interested
best regards,
Joachim
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis