I have used DWLS method in CFA/SEM because my data is ordinal (7-Likert scale) and I have therefore estimated the polychoric correlations. Hu and Bentler's (1999) fit cut-offs were developed for continuous data using the ML method. Xia and Yang (2019) indicate DWLS produces higher CFI and TLI than ML, and lower RMSEA than ML. Indeed, I have extremely high CFI and TLI and very low RMSEA.
I have searched for simulations that give some guidance on appropriate fit thresholds for the usual CFI/TLI/RMSEA fit statistics when using the DWLS method on ordinal data, with no luck yet.
I am using the psych and lavaan packages in R, which give these common fit measures.
Would anyone know of any empirical studies or simulations for appropriate fit thesholds of CFI, TLI and RMSEA when using DWLS? (Or which fit statistics are suited to DWLS?) Many thanks!
Ref: Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. https://doi.org/10.3758/s13428-018-1055-2
There are several authors such as Hu and Bentler (1999) who propose comparative indices greater than .90, RMSA-SRMR less than 0.08, according to the estimators used on
DWLS is an estimator that over-adjusts the data and tends to raise the adjustment indexes a lot, which is only recommended in complex models such as unifactorial dichotomics that have a large number of indicators
Thanks Cristian Ramos-Vera for your response that DWLS over adjusts the data. Given the literature that DWLS (or WLS for larger n) is the appropriate estimator for the polychoric correlations that are inherent with ordinal data then ML is not something I can use, so I am in search of revised cutoffs of Hu and Bentler's suggested indices when DWLS is used.
According to Tabachnik and Fidell a 7-step Likert scale can be used as a continuous scale.
Thanks Béatrice Marianne Ewalds-Kvist for the reference, there are others as well that assert the same - but they rest on the assumption of normality. It doesn't make statisitcal sense to calculate mean and SD or ordinal data (yet people do it).
My own comparison analysis using maximum likelihood produced different results than DWLS (the methods statisticians recommend), thus, continuous methods distorted my results, so how many others?
I believe that social scientists using CFA and SEM should use the analysis technique that matches the data type, so polychoric regressions for ordinal data and Pearson's regressions for continuous. In any case, this is what I advocate. I think I am finding that the recommended fit thresholds haven't yet been established for polychoric regressions. Thanks again for your response.
There are valuable info in this article; The Influence of Number of Categories and Threshold Values on Fit Indices in Structural Equation Modeling with Ordered Categorical Data
Thanks Mehmet Akif Karaman , were you referring to the Xia and Yang article by any chance? I appreciate their inference, that fit indices should be associated with the method they were developed for, and treated with caution if applied to other methods.
Hi Andrew Kemp , thanks for starting this thread. I also have the same question, although in my analysis I am using 'yes/no' data that I was told to treat as ordinal. Here's a doctoral dissertation that I found which might be helpful: https://etda.libraries.psu.edu/catalog/24901
If you have found any references on the cut-offs, I would be interested.
Li, C.-H. (2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369–387. doi:10.1037/met0000093
https://sci-hub.tw/https://pubmed.ncbi.nlm.nih.gov/27571021/
Thank you Andrew Kemp for starting this thread. I'm in exactly the same situation. For now, I decided to cite the Xia and Yang paper and simply take note of the fact that this is a developing field and fit indices for CFA with polychoric correlations and DWSL have not yet been well established. I accept slightly stricter cut-off values (CFI>~0.98, RMSEA
- Badges
- Science method