Hi,
I have questions about model selection. I fitted ordered logit, poisson and multinomial logit models on my data to choose one of them.
1) should i consider the number of significant variables in each model? i mean is the model better fit if more variable are significant? or important variables are significant?
2) I used variables based on literature review and my dataset and variables that i think important predictors. should i remove the variables that have p value higher than 0.05 and them compare the models? if not, should i interpret insignificant variables also in my paper?
3) I used brant test for testing proportional odds and test for dispersion for Poisson model and i figured that, ordered logit model and poisson are not suitable for my data and i should now go for multinomial model. should i still compare BIC and AIC of them to choose or i can based on brant test and dispersion test choose multinomial model as the final model?
thank you
1) Theory and prior research should guide what indeps to use
2) You should not remove indeps based on p-values
3) If ordered logit and Poisson are no suitable, go for multinomial
Chris :-)
Christer Thrane thank you Christer for your answer, if I do not remove the non-significant variables, is the model still valid? in most of the papers or videos that I have seen, researchers remove the non-significant variables based on the p-value.
so, before comparing based on AIC or BIC, i should consider the results of brant test or dispersion test for ordered and Poisson models, thank you
Those who remove non-significant varables do not know what they are doing, quite frankly. Such model-trimming should genarally be avoided.
Hi @Samaneh. Just to share a nice paper (https://journalofbigdata.springeropen.com/articles/10.1186/s40537-018-0143-6) clearly disscussing in details what @Christer has already argued. Mind that both backward (what you considered) but also forward stepwise regression might result in serious problems. If you need to reduce the number of variables, you might apply e.g. principle component analysis and then use in model estimation values of few components instead of values of many variables. Here you can find an example how we used that concept (https://www.researchgate.net/publication/336430233_The_Impact_of_EU_Grants_Absorption_on_Land_Cover_Changes_-_the_Case_of_Poland). Good luck with your research. Cheers. Tomasz
Tomasz Napierala thanks a lot for the useful links, actually,i do not need to reduce the number of predictors, my question was mainly about model selection, I wanted to know, should i consider which model has more significant variables? or should i choose based on model validation measures like brant test for ordered logistic models or AIC, BIC and so on.