In my study I have a time series data and I'm trying to run a binary model which includes a categorical (binary) dependent variable and nine continues independent variables. To eliminate the multicollinearity I used the PCA method. The eigenvalues of PC1 and PC2 are more than 1 and they represent 80% of the variables. After that I run the binary model using PC1 and PC2 and that shows a significant p-value for PC1 only. Now what is the next step to interpret the model variables? how can I decide which is significant and which is insignificant variable?!
Look at the stat's for your regression coefficients. P=0 isn't doing much for you see: attached. Best, D. Booth
Hello Raghad,
For your data set, the composite variable, pc1, apparently is helpful for explaining differences on the outcome variable. The variables that substantively comprise pc1, therefore, are useful as a set. ("Substantively comprise" implies the strongest loading variables, based on whatever your criterion for salience might be.)
You can't say much beyond that about individual variables. You can, of course, isolate the variables (as a set) that substantively comprise pc2 as not being particularly helpful as a set.
Good luck with your work.
As David Morse explained you can use PC1 and discard PC2.
Remember PC2 explain residuals of PC1, so if PC1 explains most of varince of your date set will be justified to employ this one.
All the best
Principal components analysis of predictors does not, in general, "eliminate" multicollinearity. Although you appear to be interested in retaining only one PC (and as you make that decision with an inferential test of the PC's association with the criterion, thus ignoring the representation of the original data via the PCs, I am not at all sure that I am in favor), a more general approach is to consider the case with multiple PCs; it applies to you as well. The PCs are themselves, by definition, uncorrelated, but if they are used as predictors, and thus weighted, the original variables are actually "double" weighted by a product of their weight to determine the PC and the weight accorded by whatever prediction algorithm used for that PC. The point here is that the PCs are artificial and that for prediction, you eventually get back to a weighting of the original variables. So, although it may not be obvious, you do have weights for your original variables that you may consider.
Now, such rank reduction does have an effect on multicollinearity, it just doesn't "eliminate" it. A good way to look at this is to see the influence of multicollinearity on prediction weights as inflating their variance. A very good exploration of that notion is Richard Darlington's PB article wherein he considered several alternatives to OLS to decrease that effect, all of which he dubbed "Reduced-Variance Regression" -- included were Stein-type, ridge regression, and regression on PCs as you are accomplishing. There is more that can be said, but this the basis.
The reference is:
Darlington, R. B. Reduced-variance regression. Psychological Bulletin, 1978, 85, 1238-1255.
- Badges
- Science topic
- Similar topics
- Philosophy
- Philosophy Of Science
- Reasoning