Hi everyone,
I am running a SEM and currently have a chi-square statistic of 5.188, with a sample size of 1765. I've read that the cutoff for the chi-square statistic should 3 or less. However, I've also been told that for large samples, a reasonable fit for the chi-square statistic can be larger than 3, although I'm not aware to what point?
Can anyone confirm this or provide me with some research to support that?
Cheers,
James
Hello James Phillip White
The chi square test is too liberal (i.e., too many type 1) errors when variables have non-normal distributions, especially distributions with kurtosis. Also, with small sample sizes (which is not your case at all) there are too many type 1 errors. For models with about 75 to 200 cases, the chi square test is generally a reasonable measure of fit. But for models with more cases (400+), the chi square is almost always statistically significant. Chi square is also affected by the size of the correlations in the model: the larger the correlations, the poorer the fit. Sometimes chi square is more interpretable if it is transformed into a Z value. The following approximation can be used:
Z = √(2χ2) - √(2df - 1)
For SEM models I think it is more reliable that they meet the criteria of most of the model fit indexes, because the x2 is conditioned by the sample size
Hello all,
while Giovane has presented one typical side of arguments, let respectfully represent the other side --for the sake of balance.
The chis-square demarks the degree of differences between the data and the model (more precisely, between the empirical covariances and the model-implied) that would be expected by pure chance (sampling error). A significant p-value, hence, notes that there are deviations beyond those expected by chance. In my experience, it is simply a matter of curiosity as well as scientific responsibility to at least take a look into possible causes or problems of the model. I had a lot of cases where this led to a eureka moment as you learn why you failed with your theory. Sometimes, this happens (and you discover a potential reason) --sometimes you don't.
The latter is no reason for despair. What is problematic, however, is to institutionalize explain-away reasons for disregarding evidence against the model. Giovane is correct with regard to the small sample size but this can easily be corrected by the SWAIN correction (Herzog & Boomsma, 2009). A large N, in contrast, does not automatically lead to a significant test, it only increases the statistical power of the test (as it should be), causing systematic but small problems to lead to a significant test. The problem is that this fact cannot be turned around, hence, you don't know whether the problems are trivial. That is, people constantly turn "large N + trivial problem --> sign. chi-square" into "large N + sign. chis-quare --> trivial problem").
Giovane is also correct with regard to nonnormality but again, there are corrections (i.e., Yuan-Bentler-correction or the Satorra-Bentler correction). And beyond that, the often experienced p-values < .000001 can hardly be explained by nonnormality ;)
When you take a look into econometrics, it is completely normal to statistically test exclusion criteria. The same goes for the literature on graphical causal models where "testable implications" are essential. The development of this explain-away culture in the SEM field is quite astonishing...As I said: No need for ideology. Just use it as a cause to be surprised and to inspect potential problems.
The fit indexes (sorry Cristian) have unfortunately not any value. The reason why they were invented was simply marketing because as Jöreskog began to sell the old LISREL software to psychologists, they constantly experienced misfitting model. In the Festschrift for Jöreskog, his companion, Dag Sörbom said "so we invented the GFI to make them happy" (real quote).
I am constantly plagued by misfitting model but it turned out as very fruitful to at least take a look.
And by the way: you seem to report the chisquare/df ratio. What is your real chis-square value, the df and the p-value?
Here is a bit of literature which is perhaps of interest:
Antonakis, J., & House, R. J. (2014). Instrumental leadership: Measurement and extension of transformational–transactional leadership theory. The Leadership Quarterly, 25(4), 746-771. doi:https://doi.org/10.1016/j.leaqua.2014.04.005
Hayduk, L. A., Cummings, G. G., Boadu, K., Pazderka-Robinson, H., & Boulianne, S. (2007). Testing! Testing! One, two, three - testing the theory in structural equation models! Personality and Individual Differences, 42(5), 841-850.
Herzog, W., & Boomsma, A. (2009). Small-sample robust estimators of noncentrality-based and incremental model fit. Structural Equation Modeling, 16(1), 1-27. doi:10.1080/10705510802561279
Kline, R. B. (2016). Principles and practice of structural equation modeling (Fourth ed. Vol. 156). New York, London: The Guilford Press.
McIntosh, C. (2007). Rethinking fit assessment in structural equation modeling: A commentary and elaboration on Barrett (2007). Personality and Individual Differences, 42(5), 859-867.
Nevitt, J., & Hancock, G. R. (2004). Evaluating small sample approaches for model test statistics in structural equation modeling. Multivariate Behavioral Research, 39(3), 439-478.
Ropovik, I. (2015). A cautionary note on testing latent variable models. Frontiers in Psychology, 6, 1-8. doi:10.3389/fpsyg.2015.01715
Thoemmes, F., Rosseel, Y., & Textor, J. (2018). Local fit evaluation of structural equation models using graphical criteria. Psychological Methods, 23(1), 27-41. doi:10.1037/met0000147
Best,
--Holger
Please check the following:
https://www.statisticssolutions.com/using-chi-square-statistic-in-research/
https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_hypothesistesting-chisquare/bs704_hypothesistesting-chisquare_print.html
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size