A colleague and I are working with a scale that, according to EFA, clearly has 20 items and only two factors (10 items per factor - with nice high loadings; the items' extracted communalities are high, also).
When we use CFA with a fresh sample (N = 360), the SRMR is good (< .05) but some of our goodness-of-fit indices (normed chi-square, CFI, and TLI) are barely satisfactory and the RMSEA is quite unsatisfactory (.96).
I wonder whether that's because we have so few factors (only the two) and 10 items on each. When I see other, successful, CFAs, they usually have more than two factors and fewer than 10 (sometimes as few as only three or four) items for each factor.
I haven't seen anything in the literature that gives me information about the "phenomenon" I'm raising - i.e, an interaction between factor and item numbers that influences model fit - particularly when the former are low and the latter are high. The authoritative sources I read seem to be based on simulated data that deal with only one parameter (either number of factors or number of items) at a time.
I'd like to hear about the experiences of other researchers with "real life" data concerning the issue I have raised here. In essence, can others get good model fit with only a couple of factors and a large(ish) number of items?
Hello Robert,
Ideally, lots of indicators loading onto few factors should have no advantage or disadvantage when compared to competing models featuring more factors, each with fewer affiliated indicators. However, let’s just consider why the former scenario might be more of a challenge:
1. It may genuinely be harder to find lots of indicator variables that reliably correspond well with each other and therefore might be shown to affiliate with a single latent variable. Consider that, with 10 indicators, that’s 45 separate relationships that have to (continue to be, with new data) strong in order to show common factor affiliation.
2. The “…may be harder to…” likely accounts for why there are so often just a few indicators in a lot of published measurement model assessments. I suppose that the trivial case of barely different survey items might be a counterexample (e.g., #1 “I like bowling,” #2 “I enjoy bowling,” #3 “Bowling is fun”), except that respondents might be driven mad in the process.
3. In the same vein, assume that the EFA resulted in overfitting (always a plausible assumption). Are the odds better of a small number of relationships holding across samples or a large number of relationships holding across samples?
4. Since CFA can, unless you’re evaluating each latent variable individually, and usually does result in correlated factors, could a “busier” model (more factors) offer more degrees of freedom for specific/unique variation to be accounted for across samples? Since the reports of many CFA analyses I’ve seen don’t address what the interfactor relationships should be (other than zero or something different than zero), I think the answer is, “yes.”
That’s my thought experiment for the day; thanks for raising an interesting query.
Good luck with your work.
Hi Robert,
the goal of modeling is not to fit the data. "Fitting" is the goal of predictive models like regression or machine learning. You can manage to fit every causal model (and factor models are causal models) by adding parameters. And I am talking not of capitalization on chance/overfitting (that adds to the problem). That is, model fit is a tool that as a use for the overall goal--to get it right.
Hence, you have to start considering whether a factor model with 10 indicators is plausible a priori. Your model implies that the sole reason why all of these 10 indicators correlate (which results in 45 correlation as David noted) is the common causal influence of a single tiny latent variable and once you adjust/control for the latent variable, all correlations disappear.
I NEVER saw a factor model fitting with more than 5 indicators, simply because various indicators impose so many verbal paraphrases that very likely change the latent you measure. Actually, there is so much research on the cognitive processes of respondents that support this worry. In my dissertation, I once read a sentence in one of the books by the Sudman/Bradburn/Schwarz group (see below) that no sane person would expect seeing the same question (albeit paraphrased) more than one time (which they called a law of communication). As a consequence, I respondents try to interpret the discriminant information in any further item, which leads to discriminant validity. As a consequence, sets of items seemingly measuring the same construct lead people to search for the details and they respond to these details.
Hence, I assume, your model is severely underfitting. If you like, you could present the covariance matrix together with the question wording and some background and we walk through a specification process....
Best,
--Holger
Sudman, S., Bradburn, N. M., & Schwarz, N. (1996). Thinking about answers. San Fransciso: Jossey-Bass Publishers.
Bradburn, N., Sudman, S., & Wansink, B. (2004). Asking questions: A practical guide to questionnaire design. San Francisco: Jossey-Bass.
Fakhar Shahzad, thank you for trying to help me. I'm familiar with the YouTube site you suggested, but unfortunately it doesn't really deal with the issue I was facing.
Again, thanks nevertheless.
thanks sir. It might be a problem in normality of data. Because of this error term of few items goes wrong.
Best of luck.
- Badges
- Science topic
- Similar topics
- Correction