Hello Kshitij,
"Step" methods were developed as "automated" techniques for reducing the number of independent variables in a model so as to end up with a smaller, "effective" set, as regards accounting for differences in the dependent variable(s). They have the advantage of being easy to program (and so, often appear in statistical software packages).
They do, however, have some substantial problems:
1. The methods are highly opportunistic, which means that the results might reflect the idiosyncrasies of the data set (and not generalize well to the population).
2. There is _no_ guarantee that a step method will identify the "best" ensemble of IVs for the variables and data set at hand (or the population).
3. Modifying the criterion for inclusion of a variable and/or exclusion of a variable can change the resulting solution.
4. In many statistics software packages, the significance tests are not appropriately evaluated (e.g., incorrect p-values).
5. Step methods remove human judgment, theory, and prior research results from the process of variable selection.
And, finally, you should always validate a step model solution using other/additional data, to assure the results have some degree of stability across samples.
Good luck with your work!
Step-wise regression includes regression models in which the choice of predictive variables is carried out by an automatic procedure.
Stepwise methods have the same ideas as best subset selection but they look at a more restrictive set of models.
Between backward and forward stepwise selection, there's just one fundamental difference, which is whether you're starting with a model:
- with no predictors (forward)
- with all the predictors (backward)
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning