For a binary outcome variable, lets say true or false OR yes or no - i have around 31 independent variables.
I tried using PCA to see if they can be reduced basis collinearity. but none was.
What is the advisable approach, should we enter one variable at a time and see if it is significant and remove the one which may be insignificant or reduce the power of the model once these insignificant variables were added.
What do the other researchers, data scientists, analyst follow.
Ideally adding one variable at a time would be great but this could be really time consuming.
Please suggest.
You can try to make correlation matrix between the independent variables. So, you can eliminate variables highly correlated.
Another option could be create a model with all variables and try to remove variables one by one, like a pruning.
In addition, in some models, like random forest, you can see the variables importance. So, even if you will use another model, you can use a model like this to check the variables importance.
I hope it helps you! Good job.
Thank you Lucas. this helps.
i have some of the variables as categorical so i cant simply run a correlation matrix. While i put all variables in the model i do see separation and glm did not converge message.
For now i was trying logistic and plan to move with decision tree and random forest.
Lucas' ideas are very good but I recommend you go one step further and try lasso with the logistic regression and boosting with the decision tree. I have attached a paper that describes these approaches. Best wishes, David Booth
- Badges
- Science topic