We performed some analysis with multivariate logistic regression (several continuos independent variables and a single dichotomous dependent variable)
A referee revising our paper asked us to show the effect-sizes of each predictors but (s)he was satisfied neither of OR’s (that I agree are unstandardized) nor of standardized B coefficient.
Which standardized effect-size is best to calculate and report for predictors of multivariate logistic regression?
Thank you!
Following Hosmer and Lemeshow (2000), my preferred effect size indicator for multivariate logistic regression is the area under the receiver operating characteristic curve. In this analysis, the predicted values (estimated logits) are the predictors and the outcome is the dichotomous outcome. Note that it does not matter whether the predicted values are probabilities or logits, as the AUC is an ordinal measure, such that the rank order is all that matters. I prefer the AUC over pseudo-r because the variance is inversely proportional to the base rate, making the proportion of variance accounted for dependent on factors related to study design.
Karl.
Massimiliano,
One approach here could be to use a dominance analysis approach to evaluating effect size of the predictors.
I have recently published a paper in Organizational Research Methods that describes the approach with ordinal and multinomial logistic regression (attached below) - but the basic idea extends to binary logistic (which is a simplification of either approach). The paper off of which mine was based is also attached that is related directly to binary logistic regression.
Take a look at these papers and see if they can apply to your analytic goals,
- joe
Article Using Dominance Analysis to Determine Predictor Importance i...
Article Relative Importance Analysis With Multicategory Dependent Va...
Ciao Massimiliano,
As already pointed out by Joseph you can use dominance analysis or also relative weight analysis (see paper attached).
Just a curiosity, did you provide the 95% CI for Exp(B) in the paper?
Good luck
Nicola
Standardized effect size isn't that great for logistic regression (or for that matter for other regressions). I've attached an overview of some of the pseudo-R square measures for these kinds of models (written for my book "Serious Stats").
More generally I think your referee is wrong to insist on standardized metrics for effect size. They don't really make much sense when the variance is a function of the mean and thus the SD use to standardize r or Cohen's d or R^2 is not constant.
On the specific issue of logistic regression the effects are probably most clearly shown by plotting predicted probability as a function of predictors (over ranges appropriate to the context of your data).
Many thanks Thom and Nicola for your replies and suggestions!
And nope, I didn’t reported the CI because actually the single predictor effects were just a secondary information. We were much more interested in an overall model significance (i.e. the global effect of groups of predictors and we used a hierarchical approach). This is exactly what the reviewer didn’t like, that is not giving a primary importance to single predictors instead of to ensembles of them, hierarchically tested with chi-square of model increases and pseudo-R^2.
In my field, and in general in medical field, it is very likely that a referee refers to the lack of 95% CI for OR when write "show the effect-sizes of each predictors".
Anyway, all the other approaches are valid.
Let us know!
Nicola
Following Hosmer and Lemeshow (2000), my preferred effect size indicator for multivariate logistic regression is the area under the receiver operating characteristic curve. In this analysis, the predicted values (estimated logits) are the predictors and the outcome is the dichotomous outcome. Note that it does not matter whether the predicted values are probabilities or logits, as the AUC is an ordinal measure, such that the rank order is all that matters. I prefer the AUC over pseudo-r because the variance is inversely proportional to the base rate, making the proportion of variance accounted for dependent on factors related to study design.
Karl.
You might consider using prediction type of configural frequency analysis :
http://www.trevipsy.ch/biposuisse/biomath/configural-frequency-analysis/at_download/file
g
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression