I have two models (Model 1 and Model 2), with different set and number of independent variables. The variables in the model 1 are selected using Stata command vselect whereas Model 2 using Stata command gsreg.
How could i determine which model is better at explaining the dependent variable?
Is it sufficient for me to just rely on the R2 or is there any Stata command/program that could decide the best model?
Can i use lrtest?
Thank You
I would consider the AIC (and/or BIC) statistic that is provided with the model fit statistics that vselect provides. This is a flexible measure that many researchers use to determine which model has the best fit. You can also get the AIC value manually by first estimating the model, and then typing estat ic
I hope this helps
Ariel
PS, there is no Stata command called gselect
You can only use lrtest if one model is NESTED within the other. That is, the set of variables in one model is a subset of the set of variables in the other model.
You should never rely on R2 for model selection. It is not a reliable enough statistic.
As Ariel said, you can use AIC or BIC and choose the model with the lowest value. Note that if their AIC/BIC scores are within about 10 of each other, the difference between the two models is considered marginal. There are a number of other similar model selection criteria you can research (DIC, FIC, BPIC, etc.).
In an ideal world, you would use a penalized regression (e.g. ridge or LASSO) or cross-validation approach for model selection. The former is possible in Stata, at least for logistic regression, but it isn't trivial. The latter is only possible if you have data that you have not used to train your current models.
Trick to AIC/BIC is they have to be using the same population. So if there's missing data and you're doing complete-case analysis, you may have an issue.
Ariel Linden: I've edited my question: Model 1 is based on stata command vselect, whereas Model 2 is based on stata command gsreg (http://fmwww.bc.edu/RePEc/bocode/g/GlobalSearchRegression.pdf)
My initial comment still stands. Run the two commands, find the best choice in models and compare the AIC and BIC estimates
Can you please help me with the stata command to run AIC/BIC in a logistic model?
you would run this command:
qui logit ...
est store m1
qui logit ......
est store m2
est state m1 m2
logistic...
estat ic
see: https://www.stata.com/manuals13/restatic.pdf
How to selection suitable ARIMA model with AIC and BIC in stata? I do not know command those.
thanks
hi Kofi, please
run the logistic regression and then run eststat ic
you will get the AIC and BIC value of the model