I did a backwards regression with various binary variables, but some of the binary variables are negatively correlated with the outcome, which would make no sense cause the correlation univariate correlation is positive. I think this is due to supression. I checked this for some of the variables (with a chi-square test), and the variables where indeed significantly correlated. But what can i do to solve this? I think the fact that they are correlated is not that odd, because for example one variable is 'presence of a disorder' while the others represent the presence/absence of different specific disorders. But how can i still do a backwards regression while correcting for those multicollinearity?
There are several opinions on backward and forward selection procedures, but as far as I know, the overwhelming majority says: Do not do it! :-) In that perspective, you only have a multicollinearity problem, which has at least four solutions:
Do noting if everything else looks fine.
Add more observations if possible
Make an index of the two variables causing the trouble
Remove the variable causing the trouble
Thank you for answering.
Wouldnt it distort my backwards regression? I also need to examine which variables have the highest variance in the outcome, so keeping those multicolinear variables would distore these findings
As I said, I see no reason to to backward selection. In my book, a model should contain the independent variables that theory/prior research suggest belong there. Then the regression should contain ALL those independent variables, if possible. No place for backward or forward selection, that is. Sometimes, however, multicollinearity becomes an issue, which then must be solved as I proposed.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression