I am working on comparing and developing pseudo R-squared statistics for logit models particularly as applied to multilevel (that is mixed or hierarchical) models. I am particularly keen on hearing about any recent developments say since 2010. The most recent development that I have found is
Tjur, T. (2009) “Coefficients of determination in logistic regression models—A new proposal: The coefficient of discrimination.” The American Statistician 63: 366-372.
Is there anything subsequent to this or indeed anything recent that compares different measures. I am aware of these two earlier comparisons
Menard, S. (2000) “Coefficients of determination for multiple logistic regression analysis.”The American Statistician 54: 17-24.
Mittlbock, M. and M. Schemper (1996) “Explained variation in logistic regression.”Statistics in Medicine 15: 1987-1997.
Thank you
Kelvyn, this recently published article might be of interest to you:
Hemmert, G., Schons, L. M., Wieseke, J., & Schimmelpfennig, H. (2016). Log-likelihood-based Pseudo-R2 in Logistic Regression: Deriving Sample-sensitive Benchmarks. Sociological Methods Research, published online before print, March 18, 2016. DOI: 10.1177/0049124116638107
Abstract
The literature proposes numerous so-called pseudo-R2 measures for evaluating “goodness of fit” in regression models with categorical dependent variables. Unlike ordinary least square-R2, log-likelihood-based pseudo-R2s do not represent the proportion of explained variance but rather the improvement in model likelihood over a null model. The multitude of available pseudo-R2 measures and the absence of benchmarks often lead to confusing interpretations and unclear reporting. Drawing on a meta-analysis of 274 published logistic regression models as well as simulated data, this study investigates fundamental differences of distinct pseudo-R2 measures, focusing on their dependence on basic study design characteristics. Results indicate that almost all pseudo-R2s are influenced to some extent by sample size, number of predictor variables, and number of categories of the dependent variable and its distribution asymmetry. Hence, an interpretation by goodness-of-fit benchmark values must explicitly consider these characteristics. The authors derive a set of goodness-of-fit benchmark values with respect to ranges of sample size and distribution of observations for this measure. This study raises awareness of fundamental differences in characteristics of pseudo-R2s and the need for greater precision in reporting these measures.
Regards, Karin
Hi!
Please check the attached link. Maybe it would be helpful to you.
Yours.
http://www.sciencedirect.com/science/article/pii/S0304407615001876
This paper might be relevant:
Nakagawa, S. and Schielzeth, H. (2013), A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol Evol, 4: 133–142. doi:10.1111/j.2041-210x.2012.00261.x
These papers from a collegue could be relevant:
1) Lila Ricci, Raúl Martínez, Adjusted -type measures for Tweedie models, Computational Statistics & Data Analysis, Volume 52, Issue 3, 1 January 2008, Pages 1650-1660, ISSN 0167-9473, http://dx.doi.org/10.1016/j.csda.2007.05.017.(http://www.sciencedirect.com/science/article/pii/S0167947307002174)
Abstract: R 2 -type measures are commonly used tools for assessing the predictive power of linear regression models; for generalized linear models this measure is based on deviances. For small samples, it is necessary to adjust R 2 for the number of covariates. This article shows that some of the adjustments that have been proposed for logistic, Poisson, gamma and inverse Gaussian models, can be expressed in a general form that covers all exponential dispersion models with power variance function (also called Tweedie models). The adjusted measure is asymptotically unbiased. Numerical results of a simulation study are presented to explore the estimator behavior under different sample sizes, different number of covariates and different population values of the adjusted deviance based measure. The performances of the adjusted measures are also investigated on a data set used by Box and Cox.
Keywords: Tweedie models; R 2 -type measures; Shrinkage
2) Lila Ricci, Adjusted -squared type measure for exponential dispersion models, Statistics & Probability Letters, Volume 80, Issues 17–18, 1–15 September 2010, Pages 1365-1368, ISSN 0167-7152, http://dx.doi.org/10.1016/j.spl.2010.04.019.
(http://www.sciencedirect.com/science/article/pii/S0167715210001288)
Abstract: An adjusted R -squared type measure for exponential dispersion models, based on Kullback–Leibler divergences, is described and it is proved that the corrected measure is an asymptotically unbiased estimator of the population value, under the null hypotheses of no covariates.
Keywords: R -squared; Shrinkage; Exponential dispersion models
Kelvyn, Do you have any suggestions for pseudo R-squared statistics for multilevel logit models?
- Badges
- Science topic