In statistics, what's the basic difference between factor analysis and principal component analysis?
Principal component analysis (PCA) is a technique used to emphasize variation and bring out strong patterns in a dataset. It's often used to make data easy to explore. A Factor Analysis approaches data reduction in a fundamentally different way. It is a model of the measurement of a latent variable. Despite the similarities, there is a fundamental difference between PCA and factor analysis: PCA is a linear combination of variables; Factor Analysis is a measurement model of a latent variable.
It has been discused before in researchgate:
https://www.researchgate.net/post/Factor_analysis_Vs_PCA/1
Dear Asit,
The simple answer to your question is
Use PCA > when your goal is to reduce observed data
Use FA > when your aim is to find latent factors.
Please let me know if the following references/sites are useful to you:
1. What are the differences between Factor Analysis and
.
https://stats.stackexchange.com/questions/1576/what-are-the-differences-between-factor-analysis-and-principal-component-analysiAug 12, 2010 ... Principal Component Analysis (PCA) and Common Factor Analysis (CFA) are distinct methods. Often, they produce similar results and PCA is ...
2. The Fundamental Difference Between Principal Component
.
http://www.theanalysisfactor.com/the-fundamental-difference-between-principal-component-analysis-and-factor-analysis/One of the many confusing issues in statistics is the confusion between Principal Component Analysis (PCA) and Factor Analysis (FA). They are very similar in ...
3. Principal Component Analysis versus Exploratory Factor
...
http://www2.sas.com/proceedings/sugi30/203-30.pdfPrincipal Component Analysis (PCA) and Exploratory Factor Analysis (EFA) are both ... Similarities and differences between PCA and EFA will be examined.
Dennis
Dennis Mazur
The previous answers are very smart. I would add just some observation:
a) PCA is the mainly used method to identify factors, but it's not the one, nor the best in several cases: often more robust identification is reached by using Generalized least squares, Maximum likelihood, Unweighted least squares, Alpha factoring, Image factoring or Principal axis factoring (http://files.eric.ed.gov/fulltext/ED449215.pdf).
b) The original Correlation or Covariance Matrix could be easily reconstructed starting by Principal Components, but it is more difficult starting by Factors.
c) On the other hand, Factors are good representations of latent dimensions of the social universe, while CP define just the best and parsimonious linear combination of observed variables.
Daniel Gutiérrez Sánchez, Efehan Ulas, Dennis Mazur and Francesco Domenico d'Ovidio,
Thanks to the researchers for extending knowledge on the afore-said topic. Regards
Aditionaly PCA has 2 usages how I see it. Firstly, it will use correlation between variables (transform them to uncorrelated), so if there is no correlation PCA not very useful in that case. Secondly, if used for reduction of dimensionality, mechanisticly by PCA you are basicaly throwing out small eigenvalues.
Think of factor analysis as kind of extension of PCA methodology; you have group of many variables you want to explain in a smaller number of factors that are actualy latent (unobservable). As it was mentioned, you will need to use PCA, or ML, etc. to estimate parameters for Factor analysis.
Factor Analysis or rather the Exploratory Factor Analysis (EFA) is the procedure used to determine the dimensionality of items measuring the same construct. PCA or Principal Component Analysis is one of the extraction method under EFA.
In addiction to my previous answer, I have to undeline that PCA has many utilizations out of the field of EFA. PCA is not just one of the extraction method under EFA, but it is often used to remove multicollinearity by a dataset without loss of information (by example, in engineering), and other purposes.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics