First of all, you should never automatically add a parameter only because it is the one that shows the highest modification index (MI) because MIs can be very misleading. Not all suggested modifications may be theoretically or practically meaningful, and you should never blindly apply suggested modifications since they may be purely data-driven and not theoretically or substantively meaningful. While MIs can point you to key portions of your model that may be misspecified or in need for respecification, I would use MIs only as one guide to better understand what may be wrong with the model as a whole.

You should rethink your model more broadly (beyond MIs) and consider ways in which the model may be misspecified (too restrictive). For example, do you have many indicators per factor and is it reasonable to assume that all indicators (observed variables) for a given factor are unidimensional? This is often a very restrictive assumption when you have say more than 4 or 5 indicators per factor (see also the point on model size below that is related to the number of indicators). Lack of unidimensionality is one of the primary reasons form CFA/SEM model misfit, in particular, when there are many indicators per factor.

Also, are there any omitted paths in the model, either in the measurement model (e.g., omitted cross-loadings) or the structural (latent variable) model (e.g., omitted direct effects, correlations between exogeneous factors, or correlations between factor residuals for endogenous factors) that may be significantly different from zero? Each non-zero omitted path may contribute to the overall misfit of a model.

To examine the sources of a suboptimal CFA/SEM model fit empirically, it can be useful to study standardized covariance residuals (in addition to MIs). The standardized residuals (e.g., OUTPUT: RESIDUAL in Mplus) tell you which observed (co)variances are substantially over- or underestimated by your model. Often, the residuals can help uncover inhomogeneities or other misspecification in the measurement model (e.g., some items within the same factor may be more highly correlated due to similar wording, polarity, or other item-specific or method effects) and/or the structural model (e.g., misspecification due to omitted paths).

Once again, residuals and MIs should never be applied in a completely atheoretical, data-driven way, but only in conjunction with substantive considerations. Otherwise, you risk capitalizing on chance.

Another reason for model misfit can be due to having a large model (so-called “model-size effect”) with many indicators resulting in many degrees of freedom. Simulation studies have shown that chi-square values can be inflated for large CFA/SEM models—leading to rejection of too many correct models. See, for example:

Herzog, W., Boomsma, A., & Reinecke, S. (2007). The model-size effect on traditional and modified tests of covariance structures. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 361-390.

Moshagen, M. (2012). The model size effect in SEM: Inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling: A Multidisciplinary Journal, 19(1), 86-98.

Shi, D., Lee, T., & Terry, R. A. (2018). Revisiting the model size effect in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 25(1), 21-40.

Correction methods are available that appear to work well in practice. See:

Yuan K. H., Tian Y., Yanagihara H. (2015). Empirical correction to the likelihood ratio statistic for structural equation modeling with many variables. Psychometrika, 80, 379-405.

Other reasons why you may get a bad fit may be related to your data structure and sampling design rather than using a misspecified or large model. For example, non-normal data can inflate model chi-square test statistics when using standard (uncorrected) maximum likelihood estimation. As a result, a proper model may be rejected simply due to a violation of distributional assumptions. For non-normal data, robust maximum likelihood estimation (such as the Satorra-Bentler correction; ANALYSIS: ESTIMATOR = MLM; in Mplus) can be used to obtain adjusted fit statistics and parameter standard errors. When using item-level data (i.e., binary or ordinal indicators), those cannot be normally distributed by definition. Special estimators such as WLSMV (DWLS) should then be used.

Another data feature that can lead to an inflated chi-square statistic is when you have nested (clustered, hierarchical, multilevel) data such as students nested within school classes or employees nested within companies. The non-independence that arises from clustered data can lead to bias in standard errors and fit statistics. Applying appropriate methods for clustered data (e.g., multilevel modeling) can help in this case.

For further information, you can watch my Youtube videos on this topic here:

https://www.youtube.com/watch?v=yUusj7laFWE&t=1s

https://www.youtube.com/watch?v=SrI9riDr1W4

Another good source of information on this topic is the SEMNET listserv (structural equation modeling discussion forum):

https://listserv.ua.edu/cgi-bin/wa?A0=semnet

In the SEMNET archives, you can find a lot of discussion on model (mis)fit and how to address it.

Christian Geiser Thank you so the detailed, clear and beginner-friendly explanation! Much appreciated.

You're very welcome. Good luck with your work!

Kudos @ Christian Geiser very well explained in such a short text!

Excellent explanation, but for my opinion there are 2 possible cases: 1) the model is primary, the data corresponding to it are secondary. 2) the data set is primary, the model is secondary. For these two cases, the order of overcoming mismatches will be different: for case 1) the model should be changed, for case 2) the initial data should be added with the model unchanged.

Pavel Dmitriev Thank you so much! Could you please explain what the "primary" and "secondary" mean here?

This is very straightforward. Let us admit that there is a temptation to use the great variety of plausible tabulated equations that can be imposed on the approximation field. The inconsistencies are what I call the primary error or the insurmountable missing sense error. On the contrary, if the data set to be approximated has limitations, for example, in time or in another dimension, I call the discrepancy between the adopted model and the observations a secondary error of interpretation. Which, provided the model is adequate, can be overcome by simply removing the registration limits.