It is well indicated that covariance-based PCA is used when variables are on the same scale (units) and correlation-based PCA when their scales are different. If these are options to undergo PCA, then why should we still worry about standardizing/normalizing the scale? And, it is commonly seen that once we found our variables are of different scales and standardized them, then apply correlation-based PCA, not covariance-based PCA. Could it still be relevant and appropriate to apply PCA of a correlation matrix? What are the conditions that we should prefer the correlation-based PCA to covariance-based PCA given standardization/normalization is possible? Is there something beyond scale difference? What if the scale of the variable is a factor?
Hello Melkamu,
I'm not sure I fully understand the context for which you are concerned about this question, and that may be important as to what the "best" answer might happen to be. What I can say is:
1. Most of the "classic" PCA and/or factoring programs with which I am familiar work from correlation matrices, not covariance matrices. On the other hand, most SEM programs with which I am familiar work from covariance matrices, not correlation matrices (I'm omitting partial least squares approaches here). One reason is that in SEM (and regression models in general) the information about scale is considered important. As well, preserving scale makes the task of comparing cases/groups with respect to location on either components or factors more straightforward.
2. In general, the variable having the largest variance will be that which gets the most weight in linear combinations. Even if one assigns equal nominal weights to each variable, the effective weights are proportional to the SDs). To avoid this sort of thing causing one or more variables in a set to "swamp" others simply due to the metric used, one would either standardize the variables individually or standardize indirectly via correlations, as opposed to covariances.
3. Would direct standardization of all included variables be any different than indirect standardization via correlation as regards the resultant solution? No.
Good luck with your work.
Correlation inherently standardizes your variables, so if they are on different scales, just run PCA with correlations.
But first be sure you do want to use PCA, because it is a specialized application that cannot substitute for more general approaches to factor analysis.
- Badges
- Science topic