How to choose variables in regression analysis?

If you consider all 20 variables as interesting predictors, collect more data. If you don't find all of them interesting, then why did you consider them at all? There should be scientific (not statistical!) reasons on which predictors you want in your model.

What is even more relevant: how did you come to your observations? Is it from an experiment? Or from a survey? If it is from a survey: how where the participants selected? By convenience or is it really a random sample from a defined population? What about non-responders? ...

Amirmohammad Almaei

Thank you Jochen Wilhelm ,

My observations are geographical units and I can't do much about them. I chose the predictors based on previous literature and data availability.

Rainer Duesing

Do you have no a priori hypotheses about the variables? Why are you collecting the data? What is the research question? I cant imagine that your research question is "I want to collect data for all variables that have been collected in other studies and look what happens if I put them together...."

Additionally, if you have a survey or not randomized trials, you should think also about the interdependencies and causal relations between the variables. It is not a good idea just to put all variables you have into one model. You may use directed acyclic graphs (DAGs) to have a look at the causal relationships and then you can determine for which variable you or must not control to estimate specific effects. But for this you need good knowledge about the variables itself.

Amirmohammad Almaei

Thank you for your answer, Rainer Duesing !

Fabrice Clerot

i would try PLS regression (maybe you can even structure the "ACP step" with some prior knowledge on the variables so as to build meaningful latents)

Pengfei Zhao

When dealing with a large number of potential predictors and a limited number of observations, it's important to select variables carefully to avoid overfitting. The rule of thumb of having at least 10 observations per variable is a good guideline, but it's not a hard and fast rule. In your case, with 90 observations and about 20 variables, you should indeed consider reducing the number of predictors.

Both Boruta feature selection and stepwise AIC are valid approaches to variable selection, but they have their limitations. Here are a few suggestions to help you choose the best variables for your Poisson/negative binomial regression:

Domain knowledge: Use your understanding of the subject matter to identify the most relevant predictors. Consider variables that have a plausible theoretical or logical relationship with the dependent variable.

Univariate analysis: Conduct univariate Poisson or negative binomial regressions between each predictor and the dependent variable. Select variables that show a significant association with the outcome.

Correlation matrix: Examine the correlation matrix of the predictors. If two variables are highly correlated (e.g., r > 0.7), consider removing one of them to avoid multicollinearity.

Regularization: Consider using regularization techniques, such as Lasso (L1) or Ridge (L2) regression, which can help with variable selection by shrinking the coefficients of less important predictors towards zero.

Cross-validation: Use cross-validation techniques to assess the predictive performance of different variable subsets. This can help you identify the most informative predictors while minimizing overfitting.

Stepwise selection: Stepwise selection methods (forward, backward, or mixed) can be useful for identifying important predictors. However, be cautious when interpreting the results, as these methods can sometimes lead to biased estimates and inflated p-values.

Combine methods: Use a combination of the above methods to triangulate your variable selection. For example, you could start with domain knowledge to identify a subset of potentially important predictors, then use univariate analysis and regularization to further refine your selection.

How to compare two groups with only two measurements?

Enhanced Yellow Fluorescent Protein (Aequorea victoria) DNA sequence?

What kind of fluid could I use for a pitfall trap that also does not invalidate molecular testing?

What's the best way to measure growth rates in House sparrow chicks from day 2 to day 10?

Irradiation data for JAPAN?

Logistical regression using SPSS for a dichotomous dependent variable, with an independent variable of 1, 2, 5 and 10 seconds?

Do we need to write protocol for a second study of a systematic review that has been registered and published?

How to increase a model accuracy ?

Do I need to do linear regression before running mediation analysis using PROCESS MACRO by Hayes ?

Sample dilution for quantitative analysis using ELISA?