HI All,
I am working on a C-SAT model. The outcome variable is 0 and 1 i.e. either Dis-Sat or C-Sat.
There are more than 100 predictor variables i have from business.
What are the best ways to exclude some of these variables. I have a process in mind which i am listing below but need some suggestions from the larger audience:
1) Dropping variables which have multicollinearity
2) Using Pearson Correlation or ANOVA basis data type
3) Dropping variables with high missing values
4) This would be too early to use any regression methods such as - forward, backward or stepwise
5) Can we also look at PCA
All the above steps that i am listing would take a lot of time, would you please advice on any other approach which helps to put only significant predictor in the model.
Thank you. Shivi
Dear Shivi,
I am a bit concerned: don't you have ANY hypotheses for your research? I mean, what IS your research question?!?! If this is not completely explorative, you should start with reasonable assumptions an hypotheses and test this at first.
Using only significant predictor (possibly chosen by a statistical model like forward selection/backward elemination...) is in my opinion the worst method of all, because it capitalizes on chance, i.e. with 100 variable you can expect a lot of correlations by chance, which in turn biases the results of such an extraction process, since it favours only deviations in one direction --> larger effects. Therefore, you can't trust the results and it is possibly not reproducible.
If you have absolutely no idea how to start, I would recommend the PCA approach and look which variables load together on one component. Think about the content of this component, make a hypothesis of it's connection to your dependent variable and test it. With this approach you will a) reduce the amount of work/calculations you have to conduct, b) reduce the type 1 error probability, and c) circumvent multicolliniearity problems, since you used this for your favor in the PCA.
These are my two cents, hope this helps a bit.
Hi Rainer thanks for the asnwer.
This was the reason as you stated above i did not went ahead straight away with the model selection techniques.
I guess my approach is fine and but take inputs from your suggestion as well where i will look at PCA.
While we talk about multicollinearity, for instance i have 2 variables - age and gender which have multicollinearity, as per the suggested approach we should build separate models at a time with one of the multicollinear variable at a time and then suggest which variable has more impact on the outcome variable. this is a robust process but a very tedious one, would you say PCA would be able to assist.
I would learn more about PCA in the interim as i am novice with this technique.
Regards, Shivi
t-SNE is useful tool for visualizing high dimensional data that steps beyond some constraints of PCA: https://distill.pub/2016/misread-tsne/ Ideally it may present you with some feature reduction opportunities.
High dimensionality may not be a particularly concerning issue if you consider modeling your particular target with some penalization or regularization function on the predictors. 100 predictors also may not be particularly large considering your number of observational units?
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA