Honestly, since you are asking for "the best criterion": expertise. Just expertise. No selection rule based solely on statistics is scientifically generally good. It can maximize some long-run fit-measure or minimize some long-run risk rate, but this does not neccesarily coincide with scientifically good models. Those can onl be justified by expert judgement.

Honestly, since you are asking for "the best criterion": expertise. Just expertise. No selection rule based solely on statistics is scientifically generally good. It can maximize some long-run fit-measure or minimize some long-run risk rate, but this does not neccesarily coincide with scientifically good models. Those can onl be justified by expert judgement.

I think that Jochen has a good point.

But when we have many variables, it is good to employ the statistical authomatic methods of selection in the first aproximation. Then we will see that some variables are unuseful, and so we may reduce the scope to consider each of the rest of the variables in detail applying common sence and expertise. CP and Stepdisc are good automatic methods and I use both always.

You can use something called best subsets. It is like a stepwise method.

If you have lots of covariates to test, you need to watch out for correlation among them. You will need to remove variables that have a VIF over 10. You also need to worry about interactions among terms in your model.

Sometimes you can use your intuition to find the appropriate terms. By using the best subsets method, you will find multiple models that fit your data. If most of these models say the same terms are significant, and those terms are not highly correlated to each other, I would go with that. I always listen to my data not tell it what to do.

I agree with Andrew. You can use the" best subsets". Also you should remove any multicollinearity among the independent variables.

Here are some comments that are critical of algorithmic methods of variable selection (including "stepwise").

http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/

http://biostat.mc.vanderbilt.edu/wiki/Main/ManuscriptChecklist -- see "Multivariable Modeling Problems"

Please check: Article Methods and Tools for Bayesian Variable Selection and Model ...

step-wise analysis