I am using SEM for my dissertation and have a sample size of 405 participants. I was planning on using CMIN/DF, CFI, and RMSEA; however, I read that larger sample sizes can cause CMIN/DF to almost always be significant (indicating poor fit). My fit is looking hit or miss depending on the fit statistic I examine so I want to make sure I'm looking at the appropriate stats - is there a better fit statistic for larger sample sizes? Thank you in advance!
Dear Margaret Bullerjahn,
in general, I would advocate for reporting all fit indices because there is no real consensus:
1) on cutoffs (in my opinion, they shouldn’t even exist) 2) and appropriateness, --> Hazard of cherry-picking
As you correctly pointed out, the chi² is sample size dependent. Some argue that you can take the ratio of chi²/df in case of large sample sizes.
Besides the chi²/df you should also look into:
- RMSEA (Root Mean Square Error of Approximation): it is similar to chi²/df but more adjusted for sample size (“cutoff”:
Although it is true that the chi-square test becomes more sensitive to misspecifications with increasing sample size, it is not "biased" by sample size. On the contrary, the test more accurately tells you whether something is wrong with your model as the sample size increases. This is expected and desirable.
"Closeness-of fit" indices such as RMSEA, CFI, etc. do the opposite: They become less sensitive to model misspecification as the sample size increases. This is not a good thing in my opinion. I would stick mostly to the chi-square test as well as standardized model covariance residuals for examining sources of misfit.
Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Structural equation modeling, 11(3), 320-341.
Thank you Ibrahim Demirer and Christian Geiser for your very helpful answers :)
Hi Margaret
Model fit is like a latent variable, you need to use multiple indicators. So the CMIN/DF, CFI, and RMSEA would be good indicators, but there are other indicators that would also be helpful such as the SRMSR, how well your proposed model fits compared to other plausible models, evidence of local mis-fit, how sensible the estimates are etc etc.
Sample size is one consideration, and we (to a certain degree) know how sample size is related to many of the commonly used fit statistics, but I try to think about model fit more broadly and in the the context of alternative, or competing, models. Although it's quite old I still love reading this book chapter
Jöreskog, K. G. (1993). Testing Structural Equation Models. In K. A. Bollen, & J. S. Long (Eds.), Testing Structural Equation Models (pp. 294-316). Thousand Oaks, CA: Sage.
Best wishes
Mark
- Badges
- Science topic
- Similar topics
- Mathematics