I'm running a logistic regression with about 200k observations, in which there is one binary predictor where out of the 200k observations there is only 4 occurrence of "1". And the coefficient I got for that predictor is sensitive to the epsilon value I choose.
For the following code I've tested epsilon = 1e-6, 1e-8, 1e=9 ... and they all give me completely different coefficient in value and in sign.
The coefficient converges for all the other predictors.
I'm guessing R uses a optimization algorithm to solve the glm and does that that the algorithm does not converge?
What is the treatment for that?
m3_new = glm(formula = input_formula,
family = binomial(link = 'logit'),
data = dat_train, trace = TRUE, epsilon = 1e-8)
I cannot reproduce the problem
> set.seed(123)
> dat
> fm1 fm2 fm3
> rbind(coef(fm1), coef(fm2), coef(fm3))
(Intercept) x
[1,] -1.389146 0.2905372
[2,] -1.389147 0.2905346
[3,] -1.389147 0.2905346
Hi Chenying,
You might be having an issue with data separation. This means you have a "perfect" predictor, which means that if X=1, then Y always equals 1 (or always 0). In this case, there is no exact solution to the maximum likelihood equation and the computer returns an approximate solution. That could explain why changing the tolerance (epsilon) affects the estimate. HOWEVER, I don't know why the SIGN would change. Maybe the response only has four 1's? Or the predictor only has four 1's?
Example, which borrows some code from Renaud:
dat
- Badges
- Science topic
- Similar topics
- Biological Science
- Kinesiology
- Running