Hello Dr. Castro,

Do you think this plot is really necessary? If you standarize your coeficients and produce a barplot, wouldn't you show the same thing?

Hope it helps,

Cheers

Hi Luiz,

Thanks for the comment. I did standardize the coefficients, however I fail to see how a bar plot can show the influence/goodness-of-fit for each factor.

Cheers

Well,

The barplot is not going to show anything special. I just thought that, if you happen to have many variables, a barplot in which each bar is the [standarized] coefficient of a variable would be good for you to vizualize your results. With regard to effects, I can be mistaken, but I think you can just look at the coefficients and the associated Wald tests. To grab a clue on the odds of each variable, just do exp(beta_i), where beta_i is the non-standarized coefficient.

Also, If there is no interaction, you can use your approach 1. If there is in fact some interaction, then it does not make any sense to look at each variable separately.

Hope it helps,

Cheers

I usually go for option (2), that is: vary only one of your explanatory variables and set all others to their means (or representative levels), then create predicted values.

I agree with Christoph.

Option 2 is what is called least square means, available in several stat packages. see for example SYSTAT GLM program. Also standard errors around the values are plotted

Hi Manuel,

Thanks for your input, but that is not what I meant.

As to me it would be better if you first go through all the steps which are required in fiting the GLM (like model checking (Omnibus, Hosmerlemshow...) and selection (AIC or Deviance...)). After that, fixing the effect of other variable and varying the variable of interest, you can plot it. For more detail, refer to Agriti, 2nd edition, Categorical Data Analysis; may be starting from chapter three. Then, use the thrid option!!

The 2nd solution is, I think, the most relevant and valid in the sense that it takes into account the whole model under consideration.

Best,

S.

I am not sure what is your final goal but relative value of standard error of prediction of coefficients or their t-test provide this information (on full GLM). You can plot them with simple bar chart or Pareto if you wish to order effects.

-- extension: Sorry, the Fisher scores in ANOVA for full GLM are even better in a Pareto plot.

Thanks Laszlo,

I know, all that info will be presented in a table, together with pseudo-r2 to express the goodness of fit numerically. My goal here is to show VISUALLY how well the model fits the data, one explanatory variable at a time.

In R, I will tend to run the full model with all the regressors then examine the respective importance of each regressor is estimated using ‘relaimpo’ package.

ref: Gromping U: Relative importance for linear regression in R: The package relaimpo. Journal of Statistical Software 2006, 17(1).

What is your objective? Are you interested in whether incorporation of a certain covariate improves your fit (i. e. the effect might be relevant)? Then you could compare AIC (or other information criterions) of the complete model with that after excluding the covariate in question.

Michael,

My objective is written in my comment above. I already did all the selection of variables, comparison of models with AIC, calculated deviances and pseudo-r2. The model I'm talking about is the FINAL model.

The second option is the most relevant in this case.

I usually plot the odds ratios with confidence interval (e.g. as a diamond plot) for all predictors. Could be useful to standardize predictors first.

I think the 2 nd option and the Rsq will show how good is your model. Try to dig more if possible on other significant variables may be there absence is creating this far numbers from observed.

I think is to do step 1 and choose those factors which have little influence. Run step 2 with means of the factors which you chose in step 1. This way the impact on the mode of those with little influence is also preserved. However, we may have to see how we can substantiate this from theory. AIC is better for choosing.

To show that your model fits well given each parameter you might use "marginal model plots" constructed for each predictor. Iain Pardoe gives an R package that uses a Bayesian approach to compute bands about the prediction (which are - I think - no real confidence bands), thereby allowing to state wether any deviation from the observed represents considerable uncertainty. In your case you might wish to show that the observed data is within those bands for each parameter in your model.

Here are links:

a paper: http://iainpardoe.com/research/02tas/02tas.pdf

and Dr. Pardoe's homepage from where you could download the R-package:

http://www.iainpardoe.com

cheers

I recommend to make a function - in R for example - to return all predicted values by the GLM equation. Use this function to plot the Observed vs. Predicted by replacing coefficients with zero, or combine them as you like it, to see how they effect prediction. In this case you will have a theoretical 45° line and multiple prediction scenarios in one plot. Maybe like this:

myf

Your item 2 in the statement of the problem looks fishy to me. If a factor has small influence then how can the predicted and observed be far apart when only the small influence factor is varied ? Perhaps something went amiss along the way somewhere in producing these plots.

For logistic regression, results are presented as adjusted odds ratio estimates from the fitted the model that includes all the independent variables. I am not clear on what you mean by plotting the observed and predicted, since your observed value is binary (i.e. yes or no; present or absent), while the predicted value from the model is the estimated logit, ln(p/1-p), or the estimated probability of the outcome derived from the logit, for a given combination of independent variables.

hi sir

I think that u should fit the model separately with each factor, calculate predicted values and plot. This produces nice-looking fits, but does not take into account the effect of the other factors in the fit.

and u can fit the model residuls vreus of each factor

Rajvir Singh

I think your item 2 is the best way. With a different point of view, you can, as Michal Fauler said, use the marginal distribution. In fact, you have, for all levels x_1:

P(Y=1|X_1=x_1) = Sum_x_2&x_3 P(Y=1|X_1=x_1,X_2=x_2,X_3=x_3) * P(X_2=x_2, X_3=x_3)

Use your glm fit for the term P(Y=1|X_1=x_1,X_2=x_2) and the proportion for P(X_2=x_2, X_3=x_3).

You should'nt compare this predicted probabilities with the observed data (binary outcome) but with the estimated probabilities of the full model (i.e. the proportions card({Y=1,X_1=x_1})/card(Omega)) or the minimal model (i.e. the proportions card({Y=1})/card(Omega)).

Notet that the mean profile (as in item 2) and the marginal distribution make sense if the the explanatories variables X_2, X_3 are really random variables and not choosed by the experimenter. In the last case, you have to interpret separately your fit P(Y=1|X_1=x_1,X_2=x_2,X_3=x_3) for each level x_2,x_3.

You should take second option which gives impact of the variables keeping other variables adjusted.

I think approach two is not helpful in what Francesco wants to present by the graphs. Approach two would be helpful if he wants to exclude certain confounding variables due to their small effect (resulting in a bad fit), but, if I got him right, this is not what he wants, since the model is already in its final version.

To show that even confounders with small coefficients contribute to reduce the uncertainty in model prediction and therefore improve the fit, marginal model plots are a good choice - or do I still get something wrong?

hope it helps...

Obviously, you are using some sort of regression. Am I right? And can you post the model?

Very good paper by Richard Williams on Stata's margins command to calculate and plot marginal effects in this issue of Stata Journal (Vol 12 No 2 308-331). Even if you're not using Stata it is a lucid and insightful introduction to calculating and plotting effects from models.

It is worth having a look at these classics by Frank Harrell

Harrell, F. E. (2001). Regression Modeling Strategies. New York, Springer.

Harrell, F. E., Jr., K. L. Lee, et al. (1996). "Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors." Statistics in Medicine 15(4): 361-387.

Frank's book describes how to do this in SPlus However, he also has the functions for R in the 'rms' package. Have a look at http://cran.r-project.org/web/packages/rms/rms.pdf and the 'nomogrami function.