Hi. I have more than 20 variables in the OLS regression model and would like to find the best model. By the best I assume it should have the highest possible adjusted R-squared, R-squared together. Also, I think it should offer the best mix of the independent variables out of these bunch of over 20 variables, so that it contains the as much significant variables as possible. Could you, please give an illustrative code for this kind of task? Maybe one which uses the "glmulti" package? Thanks a lot.
Yerassyl,
I would recommend using the "Step" function in the "Faraway" library (code: library(faraway) and then step(model)) as it utilizes the AIC criterion to reduce insignificant variables. R-squared will be always its max value using all predictors and shouldn't be considered as an appropriate measure of model fit.
Hope this helps.
Dear Armin,
My friend sent me your requested code as follows:
Automated model selection and model-averaging Provide a wrapper for glm and other functions, automatically generating all possible models (under constraints set by the user) with the specified response and explanatory variables, and finding the best models in terms of some Information Criterion (AIC, AICc or BIC). Can handle very large numbers of candidate models. Features a Genetic Algorithm to find the best models when an exhaustive screening of the candidates is not feasible.
References
- http://www.jstatsoft.org/v34/i12/paper
- http://cran.r-project.org/web/packages/glmulti/index.html
Load package
library(glmulti) All-subset linear regression using lm() based on AIC
glmulti.lm.out |t|) (Intercept) 2871.99 243.67 11.79
Dear Amir, as I understood from your answer, it is designed to find the best model using prespecified number of models. Although, this technique works, but it seems to be quite hard to implement. The reason is that I need to find the best model automatically using AIC or BIC criterion without creating numbers of possible models, which can grow exponentially as the number of variables increases. In my case, I have 27 variables in my model, and it is quite challenging to construct all the possible models. If I misunderstood somewhere, please, let me know.
Stepwise and All-subsets regression are poor choices. See the references below. Best wishes, David
http://www-bcf.usc.edu/~gareth/ISL/
https://www.amazon.com/Everyone-Advanced-Analytics-Graphics-Addison-Wesley/dp/0321888030/ref=sr_1_2?ie=UTF8&qid=1510599303&sr=8-2&keywords=jared+lander&dpID=51Jib4uNx3L&preST=_SX258_BO1,204,203,200_QL70_&dpSrc=srch
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Software
- R Statistical Package