I'm revising a manuscript in which we developed a 22-item measure that has a four-factor structure with a higher order.
A reviewer recommended that we compare three models: The 4-factor with higher order, a 4-factor model without a higher-order one, and a 4-factor model in which the covariances are set to 1.00. The explanation given was that if one sets all six covariances among the four factors to 1.00, this is equivalent to a 1-factor solution but also allows statistical comparison of Chi Square values since one of these solutions is nested within another.
The problem is that when I set all six covariances at 1.00, I receive an error message telling me that "the constraints on the parameters or the initial parameter values are bad". This is the only change from the four-factor model (no higher order), so it seems like the problem is happening because I am constraining the covariances.
Now, if I just compare the four-factor model to a one-factor model (not this nested model), the fit indices are clearly worse for the one-factor model (e.g., RMSEA rises from about .04 to about .12). So I think we can make the case that the four factor model (both with and without the higher order, but negligibly better with higher-order) are superior to a one-factor model, but I can't figure out what I'm doing wrong in the specific model recommended.
In case it matters, I'm using AMOS for these analyses.
Hi, Paul,
That is extremely helpful. Thank you! However, this brings up a follow-up question for me. From your explanation, I would have assumed that AMOS would have performed the analyses, but that that they would have resulted in poor fit indices. Instead, AMOS simply did not generate any output at all. This leads to three hypotheses for me:
1) I am simply making a mistake in setting up the model. This was the most plausible explanation at first, but I have exhausted all possibilities I can think of for mistakes I could be making.
2) Constraining the covariances in this manner leads to a logic error that AMOS cannot resolve-- regardless of the data. In other words, perhaps doing this violates assumptions of the analytic method.
3) This method might work for some data sets and some measures, but not for this data set. In other words, because this imposed condition fits the data so poorly, AMOS is unable to generate estimates.
From what I've described, is one of those explanations more likely than the others?
Steve
Paul, I hope I'm not imposing too much by asking one final question.
The only way I was able to get AMOS to analyze the model is by constraining the variance of the four factors to 1.00. It seems like this would be inappropriate, though, since I don't constrain the factor variances for the standard four-factor model, correct?
Steve
