Technically: Yes, you can (the how depends on the software you are using).

Substantially: You should not use stepwise regression. Whether you are using forward or backward stepwise selection techniques -- they all will suffer due to the problem of idiosyncrasy of sample data. I recommend to read the collection of arguments put together at http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/ and the references given there.

Additionally, you are right to suspect special problems with nonlinear models (as opposed to OLS) because they are affected by omitted variables, even when these variables are unrelated to the independent variables in the model, see Mood (2010) [the link to the reference is given below]. But because of the more serious problem mentioned above it would be ill-advised trying to technically solve this specific problem.

http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/

http://esr.oxfordjournals.org/content/26/1/67.abstract

Technically: Yes, you can (the how depends on the software you are using).

Substantially: You should not use stepwise regression. Whether you are using forward or backward stepwise selection techniques -- they all will suffer due to the problem of idiosyncrasy of sample data. I recommend to read the collection of arguments put together at http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/ and the references given there.

Additionally, you are right to suspect special problems with nonlinear models (as opposed to OLS) because they are affected by omitted variables, even when these variables are unrelated to the independent variables in the model, see Mood (2010) [the link to the reference is given below]. But because of the more serious problem mentioned above it would be ill-advised trying to technically solve this specific problem.

http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/

http://esr.oxfordjournals.org/content/26/1/67.abstract

You can use both forward and backward selection in binary regression models.

There are other options as penalized regression using LASSO, MCP or SCAD penalizations. If you are using R the package ncvreg https://cran.r-project.org/web/packages/ncvreg/index.html can be used.

It also offers the option to select the penalization parameter using cross-validation.

"If you have a very large set of potential independent variables from which you wish to extract a few--i.e., if you're on a fishing expedition--you should generally go forward. If, on the other hand, if you have a modest-sized set of potential variables from which you wish to eliminate a few--i.e., if you're fine-tuning some prior selection of variables--you should generally go backward. (If you're on a fishing expedition, you should still be careful not to cast too wide a net, lest you dredge up variables that are only accidentally related to your dependent variable." [Source: http://people.duke.edu/~rnau/regstep.htm]

Yet, the best answer for your question is probably this one (it comes from a certain Robert Butler from iSixSigma forums):

"Because the intercorrelation between the regressors affect the order of term entry and removal. Since you are approaching the “final” model from two different directions this aspect of your matrix can cause the methods to converge on different models.

A number of books recommend using just backward elimination and they have a myriad of reasons for this choice. Their preference for backward elimination over forward selection is driven by the fact that in the forward selection process a regressor added at an earlier step in the process may become redundant because of the relationship between it and those regressors added afterward. As a result, the final model may contain terms of little value.

The stepwise method (forward selection with replacement) gets around this problem by checking the status of the entered regressors at each step and, if they become redundant, allowing for their removal.

To this end, other books recommend running both backward elimination and stepwise. If the methods converge on the same model – well and good – and if they don’t then you as the analyst will have to resolve the disagreement.

I always employ both backward elimination and stepwise when examining a data set. It’s been my experience that if the data is from a DOE and none of the horses have died that both methods will converge on the same model. If one or more horses are MIA then the two methods may or may not converge to the same model. If the data is happenstance, different models with the two methods are quite common."

You can perform stepwise methods for logistic regression and they are implemented by many software. But using these stepwise methods are not recommended, and depending solely on the results of these methods would be misleading. Instead use subject matter expertise. There are many threads in RG and discussions on the net about this topic.

I think the question is: do you need to eliminate variables after all? That is: how many variabls and how many data points do you have?

If you have enough data, the best procedure is this:

1. choose variables from a priori knowledge

2. fit the model

3. validate the model by checking modelling assumptions.

If you have too many variables and not enough data, your results will probably not be reliable anyway (even less so when using automated model selection procedures).

I recommend an old paper on this topic:

https://www.jstor.org/stable/2685877?seq=1#page_scan_tab_contents

In short, the selection of variables and their order of entry (or exclusion)  should be determined based on theory (see Aneshensel's Theory-Based Data Analysis for the Social Sciences).

To support the arguments of Stefanie (thanks for the reference!) I am attaching Stata commands (file "stepwise.do") and the respective output of one run (file "stepwise.pdf") that demonstrates the nonsense of stepwise regression.

Yes, you can. There are several methods to select the good variables for model, however, there is no conventionally best method as it depends on the interest of the users.

The  backward is the best selection  technique in certain  cases such  as  recursive  system (path analysis)  and structural equation models  in which a need to bring in all variables explicitly into the regression model and test the reduction in RSQ   because of deleting a variable or block of variables at a time.

But in other cases such as yours it is better to use Stepwise method and test the significance of the increment in    RSQ  becauseof adding a variable