In Logistic Regression (as in regression in general), when it comes to check model's diagnostics, one has to check for observations with higher-than-average leverage.
From what I read in several sources, the average leverage is equal to (k+1)/N, where k is the number of predictors, and N the sample size. In literature, they propose 2 or 3 time the average leverage as threshold for "unacceptable" leverage.
What I am wondering is if, when counting the number of predictors (i.e., devising k), do we have to also count the number of levels of categorical predictors?
In other words, if we have 1 continuous predictor and 1 categorical predictor with 3 levels, k would be:
2 (i.e., 1 continous predictor + 1 categorical predictor)
or
3 (i.e., 1 continuous predictor + 2 [i.e. the 3 levels of the categ predictor minus one due to dummy coding]) ?
I hope the quastion makes sense to you. Thank you in advance for any clarification.
I guess your aim is to find influential observations which can be considered a part of model fit. You could have also a look at the residuals. I have not seen average leverage formula for logistic regression before.
Yes categorical predictors are considered as k-1 variables so you must add k-1 to degree of freedom. I add a page which explains diagnostics
http://data.princeton.edu/wws509/stata/c3s8.html
Mehmet is correct in his reply about determining model df, However it is extremely unlikely that a categorical variable would give rise to an influential observation. Thus my suggestion would be to only consider the non-categorical predictors in your search for influential observations. This seems consistent with the classic paper on the topic: D. Pregibon, Logistic Regression Diagnostics, Annals of Statistics, 1981. Hope this helps a bit. Best wishes, David
In a regression leverage depends on the value of x but not y so leverage for logistic regression in which y is a linear function of x is going to be the same as for multiple regression.
That said high leverage is not intrinsically bad - it just means that you have informative values of x. This is potentially good for statistical power. The problem is influence - which is better measured by something like Cook's distance (a combination of leverage and distance from the regression line).
- Badges
- Science topic