I recently received a comment from one of the reviewers regarding the use of modification indices (MI) so as to improve my model fit. It says that this approach generally lacks interpretability and there should be a reference to the literature regarding the specific threshold above which corrections are necessary, for example, 3.84.
In another video i saw the minimum threshold for MI to be 20.
Now I understand that error covariances should be used to the minimum. In my work I have kept the threshold limit for MI (above 20) which led to me introduce 3 error covariances. Prior to this I had also checked for low factor loadings (0.5) and also the standardised residual values (below 2) and none of the items fell in that front. The error variances were drawn between similiar meaning items from the same factor,
My original model fit as indicated by the following fit indices: χ²(164) = 393.6, χ² /df= 2.4, CFI = 0.895 , GFI=0.805, RMSEA = 0.091.
After introducing MI it was χ² /df=2.061, CFI= 0.927, GFI=.841, AGFI=.793, TLI=.914, RMSEA= 0.077
I would be grateful on your suggestions on two things:
1) Should I introduce the MI considering the model fit before that was close to average in the first place.
2) If yes, what would be the best way to address the reviewers comment and make changes to the paper so as to get a positive response.
I would really appreciate any help in this regard.
Dear Avantika,
what I now say will not satisfy you but here we go :)
Actually, you should avoid the use of MI altogether. What the MI do is to calculate the drop in the chisq value when releasing one fixed constraint to be estimated. In other words (with a pipe system as an analogy): They keep the structure of pipes, but allow opening some valves so that the pressure (misfit) can be released. By these means, you end up with a fitting albeit even more misspecified model. The best, and most often occuring case is a factor model in which the structure is misspecified in the first place but then estimate several error covs or double loadings until the model fits. But still the set of specified latent factors is wrong.
The better way would be to take a look onto the "standardized residuals"--that is the standardized differences of the specific model implied-covariances and the empirical covariances. Large numbers (> 2) show that the model does a poor job in recreating these.
I had illustrated this in an own paper (see below, only study 1 is relevant) and the famous Rex Kline has a further paper on this out. This is no easy task but perhaps the papers help.
Rosman, T., Kerwer, M., Steinmetz, H., Chasiotis, A., Wedderhoff, O., Betsch, C., & Bosnjak, M. (2021). Will COVID‐19‐related economic worries superimpose health worries, reducing nonpharmaceutical intervention acceptance in Germany? A prospective pre‐registered study. International Journal of Psychology, 56(4), 607-622.
Kline, R, B. (2024). How to evaluate local fit (residuals) in large structural equation models. International Journal of Psychology, 59(6), 1293–1306. https://doi.org/10.1002/ijop.13252
Hope that helps
Holger
- Badges
- Science method