The model including one binary outcome (0/1; incident rate ~1.2%), one main exposure, and 13 covariates. The whole model is significant and the goodness-of-fit is OK. However, model diagnostic is completely questionable (See Fig). Almost all the observations fall into the category of y=1 had residuals larger than 3. I wonder whether this is due to the low event rate of the outcome, and can anyone give me some advice as to how to deal with this problem? If the purpose of the model is to explain an exposure-disease relationship rather than to predict, can I just ignore the model diagnostic results?
Instead of logistic regression you should try Jrip - a rule-based classifier in Weka software (http://www.cs.waikato.ac.nz/ml/weka/). This algorithm will construct a set of rules for discrimination between binary outcome (grouping variable) on the basis of main exposure and covariates (independent variables). This algorithm is extremely efficient when binary outcome is very assymetric in size (you say 1.2%) - an excellent replacement for logistic regression or even Poisson regression. Try it and you won't be dissapointed, this is state of the art method.
Kind regards,
Paško
Hi,
If your data in Binary form then you have to go for Binary Logistic Regression model.
If you want to do the Binary Logistic Regression model for your data set, it is good and appropriate one. From that you can predict the influencing variables very nicely (p - value is less than 0.05 ie., Significant). You have to find the goodness of fit from the Chi-square p - value. If your Chi-Square p - Value > 0.05 then you would conclude your choosen variables are very nicely fit to your model. This is the conclusion will arrive by you.
Kinldy see the following links:
https://www.youtube.com/watch?v=Ak_t86zm_sQ
http://core.ecu.edu/psyc/wuenschk/mv/multreg/logistic-spss.pdf
http://data.princeton.edu/wws509/notes/c3.pdf
Thanks.
All the best for you.
Thanks for your comments and I am sorry for replying late.
>>Pasko,
I will try Jrip, though it is completely new to me. I wonder how many papers published have employed this software, especially in epidemiology.
>> Senthilvel Vasudevan,
I have read the materials you provided. Although my data is in binary form, the logistic regression seems not to work well due to the heavily imbalanced distribution (the low event rate). The goodness-of-fit is find (P>0.9), but I still doubt about the fitness based on the diagnostic plots. My data fitted the nonevents well with almost zero residuals, but had poor performance with the events (residuals between 3~15). So, I am not confident with the significant covariates identified.
>> Mitchell Maltenfort,
I checked other diagnostics in Stata, including deviance, standard Pearson residuals, leverage, HL dD, dbeta, HL dX2. They all suggested that my data poorly fitted observations fall into y=1.
I used poisson, probit, cloglog and negative binomial regressions to fit the data again, finding that they gave similar results to those from logistic regression.
The smooth plot (LOWESS in Stata) showed that the continuous variable fitted the nonevents well but not the case for the events.
I am not familiar with the rpart (in R), but the main exposure did not interact with other covariates (not exactly predictors, as the model was not built for prediction). Also, the AUC was only 0.6 for the model (this does not matter?).
Well, I can help you with JRIP, I'll be in my lab next Wednesday, so you can send me details via mail.
Kind regards, Paško
- Badges
- Science topic