Dear Anshul,
please note that their is an increasing literature within the SEM community who strongly criticizes the fit indices like CFI, RMSEA etc. and propose to take a statistically significant chi-square test seriously. A significant misfit does not have to imply that the causal problems are serious (in fact they may be trivial) - but the point is that any departure of the model implications from the data *beyond chance* (which the chisquare test implies) should at least be investigated. Take the metaphor: you are waking up at night as you heard "something". Wouldn't you be alarmed? Wouldn't you wanna know?
That is, if your model does not fit the data (which it clearly does not), investigate potential reasons. Start with the data. Are assumptions (e.g., multi-normal distribution) met? If not, use robust estimators (which every software nowadays provide). If the model still does not fit (which will be the case), than first start learning how a model connects with data (i.e., the covariances). Unfortunately, most textbooks do not teach this. The shining exceptance is Bill Shipley's SEM book:
Shipley, B. (2004). Cause and correlation in biology. A user's guide to path analysis, structural equations and causal inference. Cambridge UK: Cambridge University Press.
Especially, get familiar with the "path tracing rules". They are essential for understanding SEM. You can increase your knowledge extremely if you in addition understand the concepts of "conditional independence" or d-separation (which is also treated in Shipleys book). Here's a further chapter:
Elwert, F. (2013). Graphical causal models. In S. L. Morgan (Ed.), Handbook of causal analysis for social research. (pp. 245-273). Dordrecht Heidelberg New York London: Springer.
In a nutshell, the path tracing rules inform you, how a specified causal structure relates to covariances (i.e., the "model implied covariances"). If the real covariances are different (--> misfit) than the structure is wrong (given that data assumptions are met). The conditional independence issue shows this from a different perspective - that is, which covariances disappear once certain variables are statistically controlled.
For instance, a full mediation model (A-->B-->C) has the following implications: According to the path tracing rules...
o A should correlate with B (according to the path A-->B)
o B should correlate with C (according to the path B-->C)
o A should correlate with C (according to the indirect effect from A to C)
According to the d-separation criterion, the correlation between A and C becomes zero/disappears once B is controlled.
If the model does not fit that means that the d-separation assumption failed and the path tracing rule will not result in proper covariances.
Once you mastered this knowledge, take a look onto your model: Which covariances cannot sufficiently be explained/recreated?
You can know this by inspecting the standardized residuals. This is a matrix resulting from subtracting the model implied covariance matrix from the empirical matrix. Deviations in some cells mean that their is a residual correlations that should not be there and which the model cannot explain (in the above example, this could be a correlation between A and C although B is held constant).
In case your model is a CFA model, often one can come to a better model simply to inspect the question wordings. Be critical: It is plausible that a set of indicators really measure one underlying latent variable? Often misfit is simply caused by forcing a set of heterogeneous items in a false common factor model where an aggregate or composite would better be suited. Check this.
Anyway, a misfit is no "lack of success" - it is a typical result of modeling endevours. By inspecting the misfit and behaving like a detective, one often can solve the riddle. Sometimes not. But in anyway, ignoring misfit and applying magical fit indices won't help anyone (science surely not).
Hope that helps.
Holger
Hi,
SEM aims to test if the data fit the a priori hypothesized model. The hypotheses are based on the previous theory and empirical studies.
To test different hypotheses the researcher may impose different constraints to the covariance matrix.
Take into account that if a model shows a good fit does not mean in no way that your model is “the correct model” or the better. You can never prove that your model is the “true model” (for mathematical reasons), but a good fit only indicates that your model is tenable.
If the model does not fit the data, it means that there is a discrepancy between the sample covariance matrix and the reproduced (fitted) covariance matrix. Thus, you need to check where is the source of the miss-specification. The modification indices can help. Anyway, take into account that in this way you are leaving confirmatory approach to enter in an exploratory one.
The modification Indices represent an estimation of how much the overall model will decrease if the specific parameter is freely estimated. Therefore, a model that shows a good fit should produce small modification indices. However, take into account that the MI are sensitive to large sample sizes.
A golden rule in model respecification (not anymore confirmatory but exploratory approach) is that a researcher should freely (previous fixed or constrained) estimate a parameter suggested by MI only if there is a theoretical substantive basis for doing it.
Parameters should not be freed with the sole intent of improving model fit.
Every respecification of the model should be justified on the basis of the theory.
Several scholars have highlighted the problems that occur when models are specified exclusively on the basis of modification indices (see MacCallum 1986).
There is the risk that you adjust your model according to the numerous idiosyncratic characteristics of your sample. A model is useful when it can be generalized.
Best,
Gianmaria
So can we accept the values of 0.85 instead of 0.9? Do you have any references supporting the same?
Instead to accept it, I would try to identify where is the mis-specification, what part of the theory does not work, etc.
Dear Anshul,
please note that their is an increasing literature within the SEM community who strongly criticizes the fit indices like CFI, RMSEA etc. and propose to take a statistically significant chi-square test seriously. A significant misfit does not have to imply that the causal problems are serious (in fact they may be trivial) - but the point is that any departure of the model implications from the data *beyond chance* (which the chisquare test implies) should at least be investigated. Take the metaphor: you are waking up at night as you heard "something". Wouldn't you be alarmed? Wouldn't you wanna know?
That is, if your model does not fit the data (which it clearly does not), investigate potential reasons. Start with the data. Are assumptions (e.g., multi-normal distribution) met? If not, use robust estimators (which every software nowadays provide). If the model still does not fit (which will be the case), than first start learning how a model connects with data (i.e., the covariances). Unfortunately, most textbooks do not teach this. The shining exceptance is Bill Shipley's SEM book:
Shipley, B. (2004). Cause and correlation in biology. A user's guide to path analysis, structural equations and causal inference. Cambridge UK: Cambridge University Press.
Especially, get familiar with the "path tracing rules". They are essential for understanding SEM. You can increase your knowledge extremely if you in addition understand the concepts of "conditional independence" or d-separation (which is also treated in Shipleys book). Here's a further chapter:
Elwert, F. (2013). Graphical causal models. In S. L. Morgan (Ed.), Handbook of causal analysis for social research. (pp. 245-273). Dordrecht Heidelberg New York London: Springer.
In a nutshell, the path tracing rules inform you, how a specified causal structure relates to covariances (i.e., the "model implied covariances"). If the real covariances are different (--> misfit) than the structure is wrong (given that data assumptions are met). The conditional independence issue shows this from a different perspective - that is, which covariances disappear once certain variables are statistically controlled.
For instance, a full mediation model (A-->B-->C) has the following implications: According to the path tracing rules...
o A should correlate with B (according to the path A-->B)
o B should correlate with C (according to the path B-->C)
o A should correlate with C (according to the indirect effect from A to C)
According to the d-separation criterion, the correlation between A and C becomes zero/disappears once B is controlled.
If the model does not fit that means that the d-separation assumption failed and the path tracing rule will not result in proper covariances.
Once you mastered this knowledge, take a look onto your model: Which covariances cannot sufficiently be explained/recreated?
You can know this by inspecting the standardized residuals. This is a matrix resulting from subtracting the model implied covariance matrix from the empirical matrix. Deviations in some cells mean that their is a residual correlations that should not be there and which the model cannot explain (in the above example, this could be a correlation between A and C although B is held constant).
In case your model is a CFA model, often one can come to a better model simply to inspect the question wordings. Be critical: It is plausible that a set of indicators really measure one underlying latent variable? Often misfit is simply caused by forcing a set of heterogeneous items in a false common factor model where an aggregate or composite would better be suited. Check this.
Anyway, a misfit is no "lack of success" - it is a typical result of modeling endevours. By inspecting the misfit and behaving like a detective, one often can solve the riddle. Sometimes not. But in anyway, ignoring misfit and applying magical fit indices won't help anyone (science surely not).
Hope that helps.
Holger
Anshul,
I just wanted to echo Holger's advice of inspecting the residual covariance matrix. The advantage of this is that you can get a more complete picture of where in the model the misfit is located (compared to modification indices, which can be useful, but focus on a single parameter at a time).
Hi Heiko,
hope everything works fine with you!
I totally agree but would add that a second disadvantage of the modification indices is that they assume that the given structure is correct and the misfit is only due to omitted/fixed paths. If the structure is wrong (e.g., wrong directions of a path) than the MI's cannot tell you that.
That's why nonfitting factor models which are re-specfied solely based on MI always result in many error covariance (as these act as valves for the misfit that stems from the wrong structure itself.
Best
Holger
Thank you all for the esteemed support, Will work on the suggestions given above, Thanks again.
While working on SEM, al the Model fit values of CFA, GFI etc need min value of 0.9
However while working of real dataset, we seldom get these values (even after using Modification indices)
I tried to search some literature where GFI value of 0.8 or more is acceptable.
Reference given in text.................even though the values for GFI and AGFI do not exceed 0.9 (the threshold value), they still met the requirement suggested by Baumgartner and Homburg (1995), and Doll, Xia, and Torkzadeh (1994): the value is acceptable if above 0.8
Source: Baumgartner, H., Homburg, C.: Applications of Structural Equation Modeling in Marketing and Consumer Research: a review. International Journal of Research in Marketing 13(2), 139-161 (1996)
Doll, W.J., Xia, W., Torkzadeh, G.: A confirmatory factor analysis of the end-user computing satisfaction instrument, MIS Quarterly 18(4), 357–369 (1994)
The details are given http://www.ijbssnet.com/journals/Vol._2_No._14%3B_July_2011/18.pdf
Hope this may be useful for you
Best wishes
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Structural Equation Modeling