The CFI and RMSEA fit indices of the three-factor model seem to indicate a "perfect fit" to our cognitive test battery data (N=165), which makes us a bit suspicious about reporting this in a manuscript. I have already read something on this forum about the "noncentrality parameter" chisq - df. Here in our three-factor model the degrees of freedom are 24 (i.e. larger than chisq), so it makes sense that e.g. the RMSEA fit index is 0. However, we are wondering if there is anything problematic to this/if we can do anything else with this? A perfect fit in this domain seems rather suspicious to us...
Neither your 1-factor nor your 3-factor model would be rejected according to the chi-square test of model fit for an alpha (significance) level of .05. This makes me think that your study may be underpowered. A sample size of N = 165 is not very large to test a CFA or SEM model rigorously. (As an aside, the fit isn't "perfect". A perfect fit would be indicated by chi-square = 0.)
Also, many of the standardized factor loadings are quite small. The largest standardized loading is .61 (which means that the factor accounts for only about 37% of the variance in the indicator; it is even less for the remaining indicators). This indicates that the correlations among the observed variables (tasks) may be rather modest. Small correlations also lead to reduced power to reject a model, and it makes it somewhat questionable whether the indicators that load on a given factor can be seen as unidimensional measures of that factor. You would have to assume that there is a lot of measurement error variance (unreliability) and/or task-specific variance in each indicator. In fact, with at most 37% of the indicator variance accounted for by a common factor, most of the observed variance would be error or specific variance.
Is there a way to collect a larger sample?
You can follow what Christian Geiser said, If there is no way to have enough sample, I recommend you to try Partial Least Squares Structural Equation Modeling (PLS-SEM), it may be a useful approach to analyze the data. Given that the sample size is relatively small, using PLS-SEM can help address issues related to statistical power. Check the outer loadings, validity (DV, CV) and other fit indices (chisq, NFI, GoF, SRMR , Rsq, Qsq).
- Badges
- Science topic
- Similar topics
- Biological Science
- Histology
- Sectioning