stepwise selection use two procedures, the forward and backward elimination. forward inter the variables wich better indicate (best indicators) the dependent variable in order, backward enter all your variables at the beginig and then start to eliminate those which did not have significant regression coefficients. enter method enter all your variables once and give you the regression coefficient ang their significants without any ranking. the best method depends on your purpose, you can easily google it out.

Multivariate refers to  a model with a number of variates on the left hand side of any model: principal components is an example of a multivariate analysis. Multivariable refers to a model with many predictors (for example multiple regression). I assume that you mean the latter.

Stepwise selection procedures generally perform badly. Most good models are based on modelling insight and they also depend on purpose: for example pure prediction or causal analysis. Frank Harrell's book is an excellent place to start. The book by Burnham and Anderson is also worth looking at.

Neither stepwise regression nor single predictor analysis first and then choosing predictors based on this analysis are guaranteed to find the 'best' model. 

In theory you should compare all candidate models but to do so requires using a selection procedure that penalises for overfitting and oversearching.

The field is not easy and I am afraid there is not even any general consensus as to what should be done.

As I say, Harrell's book is full of good advice.

just an additional note - you should pre-define what p value you use to choose the risk factors from the univariate analysis to go into the multivariate.  On the advice of a statistician I often use p

Hi Tauqeer:

This is not a simple question to answer. I said this because when we try to get a model we have to always consider the fact that there is not just a single perfect model. We can get a bunch of models and then we have to decide about which one we are going to use.

If we want to get a good model from a technical point of view, we will decide for model 1, for instance.

If we want to get a good model from a practical point of view, we will decide for model 2, for instance.

If we want to get a good model from a practical point of view but considering risk factors, cost factor, or another kind of factors like that, we will probably decide neither model 1 nor model 2.

Forward and backward are just some methods that help us by the time we need to decide about what independents variables consider in the model. Both have some problems that must be understand before deciding which one to use for our data-set or database.

I always recommend first of all studying our data and getting the maximum possible practical knowledge about all the variables that we think could help to explain our dependent variable.

Working on this process we will get most of our problems solved from the very beginning. For instance, we will know about multicollinearity, confusion, and other important problems that must be considered from a practical pint of view. Using Forward or Backward methods not always (in fact, almost never) will help you enough in getting this problems solved if you do not start by doing the first step of modeling.

Finally, by the time you work in modeling please remember what a famous statistician George Box said about it: "............... all models are approximations. Essentially, all models are wrong, but some are useful. However, the approximate nature of the model must always be borne in mind."

From my point of view he was complete right about this topic. I support 100% his quotation.

Regards,

HA.

Dear John, thank you so much for clear and comprehensive feedback. Really appreciated

Tauqeer:

I forgot telling you to be careful with p-values selections. You are not always free to do that.

Remember that the famous p-value is implicitly defined by the time you get the size of your sample where you decide about the % of risk (alpha) you will assume. This alpha must be the same for all your study so far. You cannot change it. You are allowed to change only for greater values alpha than the original one. You must NEVER change it for lowers alpha values than the original one. If you want to do so, you have to change your sample and / or get another one but bigger in size.

So, given a fixed risk alpha the p-value decision rule state:

If p-value

Thank you so much Hector

I do not recommend using up your p values twice. (if you actually did those steps, your third step would have to take into account step 1, and this would be very complex).

If your goal is to predict one variable (binary or not) with a set of variables and you want to make the set of variables smaller on the right hand side of the equation, as previous commentators have said most methodologists argue against the forward and backward. Most argue that what you look at should be based on your theories. If you require for some reason an automated methods, lasso-type methods are well liked. There are other model selection methods also. The excellent textbook by Hastie, TIbshirani and Friedman covers these and is free on http://statweb.stanford.edu/~tibs/ElemStatLearn/. A version of this material describing which buttons to press on the computer (in R) is at http://www-bcf.usc.edu/~gareth/ISL/. The lasso page is http://statweb.stanford.edu/~tibs/lasso.html

Well, when I said alpha I mean "alpha Significance Level = P(type I error) = alpha = risk." I prefer to use "risk" because I usually work with people in the business or engineering field.

Then, we have to remember that the sample size is a function of alpha (n = n(alpha, Error, Standard Deviation)). So, when you get the n size you are assuming an alpha value (0

Hector

Thank you really appreciated, But I m still bit confuse, because you are using very high terminology, I am just a PhD medical Scholar, I only Understand simplest stat (with sorry for this stupidity) :D

Can you please guide me, in simple words?

Use the link http://www.ats.ucla.edu/stat/examples/alr2/default.htm. Find the book they use and most of the stuff for usual logistic regression model is there.  I am not sure if they go further on this topic, but this is a good book

I am sorry but I cannot help you more.

One important think to be clear about is what I just said to you in my others comments. Use the “alpha” value that was assumed by the time you got the sample. Use it as a comparison threshold always. Do not change it. If you change it, never change it for one lower value. Only is accepted (even though it does not have a real sense) make a change by one greater than the original value.

If you do not know what was the alpha number used to get the sample, then define one (5% is very common) but do not change it anymore, or at least do not change it for a lower value.

Anyway, I have to tell you that this is a very common mistake. You can easily find this error in a lot of publications from different fields and different levels. Not just in logistic regression applications but in any modeling situation or any statistical analysis where this concept is involved or it is implicated.

My best regards for you.

HA

Thank you so much Hector for you time and guidance

First you need make clear what is your target for the study?

If for prediction, you can use multiple predictors, using more, even some of them are not significant, it would get worse prediction.

If you wand identify or estimate of effect of a predictor, it is not “the more, the better”. You need go to model diagnosis process to select a suitable model.

In statistics, there is no unique answer, there is no ‘best’ model, and there is no ‘true’ model. You only can do is select an acceptable model.

Usually, the acceptable model is the one, with better and acceptable model fitting, using fewer important predictors, but all significant predictors should be considered,  easy to interpret the results.

The model selection depends on the test hypothesis and data structure. Using logistic, the outcome is probably binary in your case.  You need make sure, one subject only has one answer, yes or no.  

You need check, the collinearrity of independent factors,  all are independent without errors. If collinearity exists, more complex approach, such as PCA should be considered. The whole topic is MODEL building and diagnosis problem, there are many way to do it, according your statement I think the following will be easy. Assume the dependent variable is continuous and normal distributed as you stated

1.       Check the type of co factors, if some have a lot of missing value or typo, correct them or let them out. Then for continuous factors, check the person correlation coefficients,  if Pearson correlation near 1 or -1 among any two, one should be gone in multiple regression. For category factors, check independency of any two. if most count appear in the diagonal of the contingence table, one of the two category variables should be gone.

2.       Do univariate logistic for all factors, which you think important, to get overview.

3.       For categorical predictor, compare the effects from different level, if necessary, you may need regroup them. If  it is ordinal, you and showing the effect (log OR or Log Odds) is monotonic, you may assign score and treat it as continuous

4.       Do PCA analysis to see if there are still multicolinearity among the independent factors, if some eigen-value is near zero, you may drop one of them, or define a new factor (transformation) .

5.       If the sample size is large enough, say 10 (at least) times higher of the unknown parameter number, you can  do multiple regressions, you may use auto select option, such as forward backward or best, which will select independent factors for you.  However, you should use the software selected model as one choice, you select several according your knowledge about the outcome and predictors.

6.       Number of parameters: interception: count 1, continuous factor, each counts 1, categorical factor with level of k, count as k-1.

7.       Check interaction among the independent factors, the interaction among two quantitative predictors, means there is joint effect; the effect of one factor varies across the level of another factor. If the interaction between an quantitation factor and a category factor, say gender, it means the effect of the quantitative is different between males and females.

8.       You may use Akaike’s information criterion or Bayesian information criterion to check the model fitting, the small of those, the better.

...

Mr. Li 

Thank you so much for your guidance

There are two approaches to be used when modeling relationships using regression analysis. This depends of what you intentions are. You can either be :

The first requires to collect data on potential confounders and correctly model the causal chain between factors to prevent over-adjustment. This is a theoretical construct and should rely on pre-existing observations. 

The second is a bit more tricky as we do not need to provide a measure of the strength of the association but we need to optimise our model to correctly predict an event. The best way around this is to plan two separate studies. One to derive your model, and a second to validate it. This lets you work around your data and build an optimal model for the first dataset. To measure the stability of your model on this data, I would suggest using bootstrapping (internal validity). You can then apply the deviated model on the second sample to verify that the model holds and that you did not construct it on random error.

To prevent this from happening, there are a few principles to respect:

• Use large samples to derivate your model.
• Only use known factors that are associated to the condition you are investigating.
• Combine factors that are correlated one to another to prevent collinearity.
• Use cut-off values or categorization that are clinical relevant.
• Favor factors that are easy to collect in the future (practical aspect).
• Only stratify if there is prior existing evidence of interactions.

I hope this helps...

follow