Dear colleagues,
I am applying the PCA to political and institutional variables to create an index and use it in the regression analysis as a dependent variable. However, the variables which will form the main components contain different measurements. For example, while control of corruption ranges between -2.5 (weaker) and 2.5 (stronger), freedom of press ranges between 0-100 and if the value is higher, it shows fewer degrees of freedom of the press. So, I am in a loss to understand if this difference creates any hardship to PCA to produce a valid index. In other words, is it a problem for PCA if one variable implies higher success as the values of it get higher, while the other variable shows higher success as the values of it get lower? What should I do in this case? Thank you beforehand.
Best
Ibrahim
Check if the bivariate scatterplots are linear or at least monotonically increasing or decreasing.
Check if the bivariate scatterplots are linear or at least monotonically increasing or decreasing.
Run a PCA but turn all your variables into Z-scores, then compare it to your original to see if scaling matters
Are you calculating your loadings (eigenvectors) from the correlation, covariance, or some other matrix?
Also, there is principle component regression that does what you are doing, but there are other methods for dealing with having lots of predictors (like the lasso).
I guess the main issue is if you have any guess about how you think all the predictor variables will relate among themselves and with the response variable, or is this more a fishing/exploratory exercise?
Dear Daniel Wright
I am calculating the loadings via a correlation matrix. My predictor variables are highly correlated (positively) with each other. However, it is exploratory when it comes to dependent variable in my final regression equation to predict it by the regressors.
Dear Ibrahim Niftiyev ,
In practice, there should be no problem in using variables with different measurements especially if such variables have reasonable correlations. However, due to the property of PCA that it presents principal components (PCs) as linear combinations of original variables, different units of measurement in variables used may cause some undesirable features on the results, eg. variables whose scores/loadings might demonstrate a complex structure. Although, from a mathematical standpoint, there is nothing innately wrong with linear combinations of variables with different units of measurement (eg. widespread use linear regression), the fact that PCA is defined by a criterion (variance) that depends on units of measurement implies that PCs based on the covariance matrix may change if units of measurement on one or more of the variables change (unless if all variables undergo a common change of scale). To overcome this undesirable feature, a common practice is to begin by standardizing the variables in the dataset. But due to the fact that you calculated scores (loadings) via a correlation matrix (which is a covariance matrix of a standardized original dataset), then there should be no problem – in my view.
All the best!
Reference: Jolliffe, I.T., & Cadima, J. (2016). Principal component analysis: a review and recent developments. Phil. Trans. R. Soc. A 374:20150202.
http://dx.doi.org/10.1098/rsta.2015.0202
Ibrahim Niftiyev ,
Herewith attached the article for your easy of reference - Section 2(C).
I agree with Ette Etuk, I prefer to have all loadings in positive value.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis