I found that from litterature it is not clear which multivariate method gives the best predictive power (testing with independant data): 1) standard multiple regression, 2) step-wise multiple regression or 3) multiple regression using the Akaike'sinformation criteria(AIC) ?
None of the above. I would look at bagging, boosting, lasso, or elastic net for variable selection when building a predictive model . These are much more stable than things like stepwise. References are given below. The paper, RF4 gives some examples of their use. Best wishes, David
https://www.amazon.com/Computer-Age-Statistical-Inference-Mathematical/dp/1107149894/ref=sr_1_1?ie=UTF8&qid=1485061947&sr=8-1&keywords=efron+hastie
https://www.amazon.com/Introduction-Statistical-Learning-Applications-Statistics/dp/1461471370/ref=sr_1_fkmr0_2?ie=UTF8&qid=1485061947&sr=8-2-fkmr0&keywords=efron+hastie
https://www.amazon.com/Everyone-Advanced-Analytics-Graphics-Addison-Wesley/dp/0321888030/ref=sr_1_1?ie=UTF8&qid=1485062184&sr=8-1&keywords=r+for+everyone
I agree with David Eugene Booth, but would like to add something.
It's often better to use logic or theory than just blindly follow a model selection process.
Imagine you are modelling some pollutant load in a river. Among IV's you have both recent rainfall and discharge of the river. Both are correlated to the DV, and correlated to each other.
There may be reasons why you want to include one and not the other. For example, you may want to include recent rainfall because it easy to get this data for any location. On the other hand, you may prefer to keep river discharge because you want this model to be used at sites with gauging stations, and you feel this would give better site-specific results.
The statistical advantage of one IV over the other may be dwarfed by practical consideration.
In a write-up, it may be difficult to explain this logic, tho.
Salvatore
It is obvious that a knowledge of the field is required prior to use statistics which often just confirm certain things (but sometimes allow you to consider new predictors). At the level of research we are involved, I do not thing that people just blindly follow model selection process (except if you don t have a clue of what you are doing...). But I agree with you that in a write-up it may be difficult to explain. Actually I believe that model selection is an art and a science at the same time where also experience not only logic or theory plays a role.
It really depends on the problem at hand, the nature of the data and the purpose of the model. The options you provide are not so much related to predictive performance, they are related to model selection. If your goal is prediction, I agree with David that you should be looking at a method like bagging, boosting, random forests etc. It would also be important to devise a scheme of training and testing sets, so as to avoid overfitting. If your goal is model selection, there are many options besides stepwise, AIC etc (eg: Bayesian Variable Selection).
- Badges
- Science topic