They are similar in many ways. PCA is an approach for only data reduction to get indices. But FA ia an approach for create measurement models.

If you want to choose correct one:

1. If you assume or wish to test a theoretical model of latent factors causing observed variables ---- Run factor analysis

2. If you want to simply reduce your correlated observed variables to a smaller set of important independent composite variables ---- Run principal component analysis

Hello Murat,

In addition to Hakan's good advice, please allow me to elaborate a little.

The chief distinction between PCA and FA is that all the observed variation in a data set is presumed to be common variation in PCA, while FA (or, common factor analysis) distinguishes between common variance (that which is shared among variables), specific variance (that which is specific to an individual variable), and error variance (that which is due to measurement error).

As a result, unless you have a substantial number of variables in the data set, PCA tends to bias variable-component loadings (correlations) higher in magnitude relative to FA. That can result in differing interpretations of both structure and salience (whether a variable does or does not affiliate with a component/factor).

Here's a brief, very readable overview of some of the differences as applied to a data set:

https://stats.idre.ucla.edu/spss/seminars/introduction-to-factor-analysis/a-practical-introduction-to-factor-analysis/

Good luck with your work.

In basic SPSS version (without AMOS), you can run from the menu only Dimension reduction/Factor analysis, where in the output table it sais it is actually PCA. In AMOS, an addition to SPSS, you can run confirmatory factor analysis (a "real" factor analysis) for creating models. Hope this helps.

Adding to what has already been said. PCA is a (rather crude) data reduction technique which produces a set of uncorrelated (orthogonal) components. Factor analysis can be much more nuanced and allows comparison and testing of underlying (potentially correlated) structures trying to get a better picture of reality in conditions of measurement uncertainty. In general, if you are interested in understanding you should be using factor analysis.

Moroever, there have been recent major developments in factor analysis, see for example

Article Exploratory Structural Equation Modeling: An Integration of ...

Here is a recent application that evaluates a number of alternative models of mental illness and points to recent developments of the FA technique:

Article Understanding the population structure of the GHQ-12: Method...

Thank you everyone, I found here the answer. https://www.youtube.com/watch?v=abQA0n83hBQ&lc=z23bczq5murpudse5acdp435r0bsleedddnxojlfcg1w03c010c.1574518268999375 Details on comments

In practice we choose principal component analysis as method for pca but another way we choose principal axis factoring for fa. (spss)

Factor Analysis is the procedure to determine the usefulness of items measuring certain construct while PCA is one of the extraction procedure in FA process.

Murat Tuna. Thank you for the site about PCA. It's quite good.

Apart from that, I notice that you say you have used principal axis factoring (PFA) with factor analysis (FA). If you use FA, however, I would recommend that you consider maximum likelihood (ML) as an alternative method of extraction. The following site might be helpful with regard to that:

https://stats.stackexchange.com/questions/50745/best-factor-extraction-methods-in-factor-analysis

Some time ago, ML was highly recommended to me by a statistician whom I respect. In some of our research, in which our data were "healthy" (e.g., no evidence of skewness or kurtosis, and good spread of scores on the items), we obtained different results with PFA and ML extractions, and the output using ML extractions appeared to make more sense.

Zainudin Awang Could you maybe elaborate more on this? I struggle with understanding how PCA is different from FA but still its combined as it is an extraction method? See my question I posted.

Thanks in advance!

Why you make yourself difficult? FA is the procedure while PCA is one of the extraction method under the FA procedure. So there is no issue at all to find the difference between FA and PCA.

Mayumi - Finding the difference between FA and PCA is just like finding the difference between USA and California.

Zainudin Awang, I wonder whether you might be confusing PCA (principal components analysis) with PAF (principal axis factoring) because PCA is not really factor analysis, and PAF one of the extraction methods within factor analysis.

In other words, PCA runs "parallel" to factor analysis (they're different things, like Canada and the USA - to use your simile), and PAF is within factor analysis (e.g., California being within the USA, again to use your simile).

I hope that's helpful.

Zainudin Awang , I don't understand your California to USA simile (I had written metaphor, thanks @Robert). As you have seen from reading the previous comments, PCA takes data, and reduces the dimensionality by combining creating weighted combination of the variables (you choose as few dimensions as useful). EFA creates latent variables, and a weighted combination of these are used to approximate the variables. They both can take a cor/cov matrix and produce variables that relate to the observed variables, but how they do this is different. One is not a subset of the other.

PCA is the simplest factor analysis (extraction of eigenvectors, no rotations, no transformations). It simply looks for each subsequent main component that represents the largest variance.

Factor analysis, on the other hand, explicitly attempts to model the underlying "latent variables". Although it also (usually) starts with the main components, they are rotated in order to improve interpretability. The typical objective of factor analysis is to identify variables that are related to each other and to separate them from each other (a form of clustering of variables); rotations help to improve this interpretation.

However, from a practical point of view, PCA is observational and FA is a modelling approach.