I'm getting more and more confused the more I read about variable selection in regression analysis, hopefully you can guide me.
We are trying to identify risk factors for developing pulmonary complications after abdominal surgery and I'm now building the regression model but I don't know how to handle previously known risk factors that aren't significant in my cohort. I started with univariate analyses of all the potential risk factors and added only those with a p value < 0.1 to the multivariate model. Smoking is an example of a variable that isn't significant in our cohort but is a well established risk factor for developing pulmonary complications in the literature. Should I add smoking to my model or not? I can't seem to find any answers in other articles since they all use different variable selection processes and I see that many researchers don't motivate their choice of variables very well.
Also, how do I interpret OR 1.0 (CI 1.0-1.0) and p value < 0.05? It is significant and the CI doesn't include 0 but it also says that there is no increased risk. Should I still add it to the multivariate model?
Are you talking about a logistic regression there when you say OR 1? An OR of 1 would say that it is 50/50, therefore it has no effect.
With respect to including Smoking or not, if you know full well it is a risk factor (paper after paper suggests that), I would keep it in there, but that's me. If your goal is to publish and you know that is a factor, it's likely best to keep it as a reviewer is going to ask about the obvious factor omitted.
You might consider dominance analysis as well. If you look at dominance analysis, it gives you insight as to which variables you add in what order. Dominance comes in three forms: complete, partial, and general. Complete dominance says that this variable always impacts R2 in a bigger way than this other variable, independent of all other model terms in the space. Partial says that within the space of models with X number of variables, it is better. General is that the average contribution is higher. This might give some insights. I would be curious as to how smoking plays into that.
Another way to think about it is the predictive capacity of the model. How big is your dataset? If you have a big data set, you could peel some of that data set off and use it to test the model against. Try modeling with and without smoking and see if your confusion matrix is better with or without it (you don't even necessarily have to split your sample to do that, but it avoids a confirmation bias potential). That can give you some justification in the choice as well.
Hope that helps!
Yes, I completely forgot to mention. It is a logistic regression model that I'm working on. Thanks for the helpful advice!
Our sample size is 417 patients. The statistician we consulted recommended the same thing, splitting the sample into two subsets and validating the model on the second one, I had forgotten about that. Thank you!
I would suggest using the LASSO method for variable selections. This method can help you to reduce the model's deviance and increase the model's accuracy. In addition, It does not affect by multicollinearity issues and sample size. As we know to use stepwise logistics regression needs 10 observations for each variable.
on the other hand, there are two to determine the variables of the model: 1 based on the literature and expert.
2. using statistical methods
Perhaps these will help: Preprint Boosting and Lassoing Cancer Risk Factors: New Prostate Canc...
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating | Ewout W. Steyerberg | download (b-ok.cc)
Best, David Booth