I tried to find out why the deviance in GLMs is defined as -2*log(LR) (with LR being the likelihood ratio). Why the factor -2? Often, authors state that this way the deviance for a normal model with identity link equals the residual sum of squares, but this should not be the reason but rather a consequence.
I found that "under some regularity conditions", the deviance has a Chi-squared distribution. Again, I dont think that this should be a reason but a consequence. (btw: I would be glad if someone could explain what conditions these are).
Is there any *logical* reason why the deviance is calculated as -2log(LR) and not, for instance, as -log(LR) ? Or, to put it the other way around: what does the *squared* ratio of the likelihoods (L[sat] over L[restr]) tell us (of which the log is the deviance)?
A good reference on deviance in generalized linear models is the classic of McCullagh and Nelder (1989), pages 118-122. In particular, there is a discussion of the two major assumptions involved in the asymptotic distribution of deviance as a chi-squared variable. Using -2 as the scaling factor has several desirable consequences including several that you have already mentioned. It is no more complicated than -log(LR), which does not t have any of the "nice" properties associated with -2log(LR). So, why would you not use it?
The value -2 lm + 2m (Akaike' s information criterion) is used to adjust the logistic regression model. In this model lm is the maximized log-likelihood and m is the number of parameters. The minimum value for -2 lm + 2m is 0 and corresponds a perfect fit (likelihood =1).
Christensen (1990 *) said: " The likelihood ratio test statistic is G2 = −2[log L(m0) − log L( m)], where ˆm0 is the MLE of m under the assumption that H0 is true and m is the MLE under the “unrestricted” model ".
* Christensen, R. (1990)Log-Linear Models and Logistic Regression. Springer-Verlag New York, Inc., pp. 332-336.
In Wilks 1938 "The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses". The Annals of Mathematical Statistics 9: 60–62.(http://projecteuclid.org/DPubS/Repository/1.0/Disseminate?view=body&id=pdf_1&handle=euclid.aoms/1177732360) you can see how -2log(LR) can be interpreted asymptotically as the squared 'distance' between the maximum likelihood estimate of the parameters and their values under the null hypothesis. That is the same interpretation that the Wald test and both, Wald test and LR test are asymptotically identical.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics