I am working on a multiple regression model with quadratic terms. Once I've got the first multiple regression equation, I checked if adding the quadratic/cubic terms could increase my Adj r-square.
But I have a question. Since I am using backward elimination, can I load all of them (12 terms) and remove the less significant (as backward elimination does) proceeding until I get the equation?
In general this approach (stepwise forward and backward) is not recommended. This is covered in several RG answers so the search you made before adding another question should have found these.
BACKWARD STEPWISE REGRESSION is a stepwise regression approach that begins with a full (saturated) model and at each step gradually eliminates variables from the regression model to find a reduced model that best explains the data.
Federico Roggio you are correct. You need to load all and remove the less significant factors
Uthayarajan Raman No, not really. While stepwise regression or elimination can is very useful in exploring the data, then don't do it on the p-value, but on the R-squared or something else (also note some suppressor effect, etc., but I take this naively for granted).
Focusing on stepwise regression using the p-value does not focus on what best "captures" (not explain) the data, but on p .05 and address this. Why should you remove them because p > .05, what is the rationale behind this?
Some books (I think it was Zuur et al. (2009) "Mixed effects models and extensions in ecology") describe this and I have no idea why. There is no clear explanation for it. Use p when you address a hypothesis, not when exploring, unless you somehow are exploring with p (but then drop the whole stringent cut-off and "significant" statement). You are just as likely to capture some noise or a signal of another variable in your data by doing so, this is not a hypothesis.
I am just curious, and this might be off-topic, by why is stepwise regression and removal of variables from a model normally performed on p-values. Hence, what rationale lies behind this approach?
Wim Kaijser and Uthayarajan Raman, thank you for your interpretation. If stepwise regression (or elimination) is not so worth it, how should I conduct a regression?
Daniel Wright, I obviously searched for other questions but, as Wim did, a solution was requested, not a personal opinion.
Sorry, you are right, but: Article Taking a Chance in the Classroom: Top 10 Reasons Not to Drop...
Ideally use theory to decide what to have in your model, but if you want automation many people use lasso (and lasso-like) approaches. See https://web.stanford.edu/~hastie/StatLearnSparsity/index.html
I should add there is some discussion in that book about how you can get some p-values when using that procedure that are meaningful (well, as meaningful as any p-value, which some would question). These are not the ones printed in stepwise procedures!
Daniel Wright thank you, I downloaded the book and I'll search for your suggestion, really appreciated!