I want to understand the differences between FA and PCA clearly.
I am more confused by what Stata is doing with the "principal-components factors" option in the "factor" command. I get different results (loadings) for "PCA" and "factor PCF" even when I restrict the number of components/factors to be the same for each procedure.
Hello Chetna,
This question has appeared several times before in researchgate.
Briefly:
1. PCA presumes all variance in a data set is common variance. CFA (common factor analysis) only works with the "reliable variance," that which is shared among the set of variables being factored. CFA proposes that total variance includes: (a) common variance; (b) unique variance (which may be further split into variance specific to a given variable and error variance).
2. PCA makes more sense when the goal is data reduction or replacement of original variable scores with a (smaller) set of linear composites which are independent of one another. CFA makes more sense when one proposes that there are one or more latent variables which influence or affect the observed scores on the variables being factored, and wishes to identify plausible structure(s) for a given data set.
3. If applied to the same data set, there is a tendency for estimated variable-component loadings (via PCA) to be higher than estimated variable-factor loadings (via CFA, using principal axis extraction as a comparison). This could well cause different analysts to make different claims about structure.
4. Either method can be further subject to rotation in space, usually in order to simplify explanation of the associated components/factors. Prior to rotation, however, the first extracted component or factor will be the one which accounts for more of the variation in the data set than subsequently extracted components/factors.
More could be said, but I believe this should be sufficient to convince you that the results can (and often are) different.
Good luck with your work.
Hey Chetna
David has provided you with a great reply - I don't have much to add.
PCA and FA are very similar in many ways, as they are both latent variable models, but in one important way they are completely different - opposite in fact! In FA the items, or variables that you have measured, are considered to be dependent or predicted by the latent variable(s). In PCA the direction of cause is reversed - the items, or variables that you have measured, are considered to be predictors of the latent variable(s).
As David noted, this type question has been asked in different ways by many people in RG. I don't mind this because it gives me another opportunity to suggest that people read, what I think, is one of the best papers in psychology that has ever been written. This neatly describes the distinction between factor analysis and PCA, and much, much more besides.
Bollen, K., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110(2), 305–314. https://doi.org/10.1037/0033-2909.110.2.305
You can get a fulltext on RG
Best wishes
Mark
Both methods are implicates for reducing the dimensions of content or data. But, PCA does that under the impediments of orthogonality and maximum variance by identifying a linear set of sources that facilitate the reconstruction of the observed data. In the case of the factor analysis, a latent variable—which cannot be measured—and the observed variables are linked in a cause-and-effect relationship. You can find Both methods in modern time statistical software.
- Badges
- Science topic
- Similar topics
- Mathematics