I have predicted an ecological variable using OLS regression which showed the model accounts for more than 72% of the variance in the dependent variable (DV). However, I am also interested in which covariates have much impact on the DV. But I found that some of the independent variables are collinear making the Variance Inflation Factor > 10. What is the solution to the multicollinearity problem both for the continuous variables Temp and Vapor ( Drop one? ) and the categorical variables 1-8 which have very high VIF?
Variable Coeff. Std Coeff. VIF Std Error t P -Value
Constant -0.228 0 0 0.086 -2.644 0.008
Precipitation
Well,
In this case I would suggest you continue with Ridge Regression which introduces bias for the parameter coefficient estimates but reduces variances and can be used when multicollinearity in a regression model is present.
As in the case of an OLS regression, in case there is multicollinearity, the design matrix X is not full rank, hence the matrix t(X)X could not be invertible hence there is no unique solution for beta(OLS).
In Ridge Regression, we introduce a shrinkage factor lambda, hence for any design matrix X, the quantity t(X)X+lambda I is always invertible hence there exists a unique solution for beta.
You can read more about it here:
http://web.as.uky.edu/statistics/users/pbreheny/764-F11/notes/9-1.pdf
OR
http://www.m-hikari.com/ijcms-2011/9-12-2011/rashwanIJCMS9-12-2011.pdf
Ridge regression could be easily fitted in R, using, for example, glmnet package.
Thao, Thank you for explaining how i should proceed.I also had a look at one proposal for multicollinearity . What do you think of it since it is not a widely known procedure?
https://www.researchgate.net/publication/254297969_A_simple_way_to_deal_with_multicollinearity
I also realized that one OLS assumption is independence of error terms; while my error terms have very small spatial autocorrelation, the modeled and actual response response variable are spatially autocorrelated. How can i solve both spatial dependence and Multicollinearity problems.
For multicollinearity I still recommend ridge regression. There are 2 possible remedies for autocorrelation. See Neter, Wasserman and Kutner, Applied Linear Statistical Models, 2nd edition, sect 13.4. Sect 11.4 covers ridge regression. Best wishes.
Are you allowed to elaborate on what the dummy variables are?
From the looks of it, you are studying plant growth.
Something that you can try too is, use a "best subsets" or "stepwise" regression model. That will eliminate a lot of the "insignificant" variables from your model. It should help with the colinearity too.
Both of these methods will try different combinations of the variables. If you find that both methods give you nearly the same model, you can feel confident that those are the variables that belong in your model. If they are very different, that is another issue.
A dummy variable is a "present' (1) or 'absent' (0) representation for categorical variable like land use - land cover.Plant growth or accumulation of biomass also depends on plant type since a forest and a cropland does not have the same biomass.
Can't recommend all subsets or stepwise as they have been shown to be unstable. lasso or elastic net would be a better choice for variable selection. Details in the paper attached. Elastic net will allow you to do ridge regression as well. See Lander, J, R for Everyone, Addison Wesley, Chapter 19. Best wishes.
- Badges
- Science topic
- Similar topics
- Biological Science
- Molecular Biology
- Biomolecules
- Nucleic Acids
- DNA