Check if the bivariate scatterplots are linear or at least monotonically increasing or decreasing.

Check if the bivariate scatterplots are linear or at least monotonically increasing or decreasing.

If increase in one variable and decrease in another variable increases the success involved, the two variables must be inversely related. The use of such variables will not affect the PCA as defined and applied.

Run a PCA but turn all your variables into Z-scores, then compare it to your original to see if scaling matters

Thank you Dear Ette Etuk Andrew Robinson Pradeep Paraman

Important in diagnosis

Are you calculating your loadings (eigenvectors) from the correlation, covariance, or some other matrix?

Also, there is principle component regression that does what you are doing, but there are other methods for dealing with having lots of predictors (like the lasso).

I guess the main issue is if you have any guess about how you think all the predictor variables will relate among themselves and with the response variable, or is this more a fishing/exploratory exercise?

Dear Daniel Wright

I am calculating the loadings via a correlation matrix. My predictor variables are highly correlated (positively) with each other. However, it is exploratory when it comes to dependent variable in my final regression equation to predict it by the regressors.

Dear Ibrahim Niftiyev ,

In practice, there should be no problem in using variables with different measurements especially if such variables have reasonable correlations. However, due to the property of PCA that it presents principal components (PCs) as linear combinations of original variables, different units of measurement in variables used may cause some undesirable features on the results, eg. variables whose scores/loadings might demonstrate a complex structure. Although, from a mathematical standpoint, there is nothing innately wrong with linear combinations of variables with different units of measurement (eg. widespread use linear regression), the fact that PCA is defined by a criterion (variance) that depends on units of measurement implies that PCs based on the covariance matrix may change if units of measurement on one or more of the variables change (unless if all variables undergo a common change of scale). To overcome this undesirable feature, a common practice is to begin by standardizing the variables in the dataset. But due to the fact that you calculated scores (loadings) via a correlation matrix (which is a covariance matrix of a standardized original dataset), then there should be no problem – in my view.

All the best!

Reference: Jolliffe, I.T., & Cadima, J. (2016). Principal component analysis: a review and recent developments. Phil. Trans. R. Soc. A 374:20150202.

http://dx.doi.org/10.1098/rsta.2015.0202

Ibrahim Niftiyev ,

Herewith attached the article for your easy of reference - Section 2(C).

Thank you so much dear Elvis Munyaradzi Ganyaupfu

Ibrahim Niftiyev ,

It's a pleasure colleague

I agree with Ette Etuk, I prefer to have all loadings in positive value.