Hell everyone.
Recently I am modeling a logistic regression. The outcome Y has 40 subjects but only 10 events (Y = 1). I hope to figure out whether predictor A is associated with Y after adjusting 4 variables. So, I have included 5 variables as X . I know according to the EPV rule, this logistic model could only have 2 variables. But adjusting the other 4 covariates is essential and after adjusting them, A turns out to be significant in the model which is good.
However, the odds ratio CI is very large [1,220]. Then I use penalized glm ('logistf' or 'brglmFit' in R), the CI turns to be [1,35]. It's better, but still to wide. I'm afraid too wide CI is not good to prove the reliability of my results.
Could you please give me any suggestions about this situation? Million Thanks!!
(all the covariates have been standardized)
If all of the outcomes are the same why do you need regression at all. Rethink this whole project and read the attached paper .
To summarize:
- You have n=40 observations
- There are 10 events (Y=1) and 30 non-events (Y=0)
- Your focal explanatory variable is A
- You are adjusting for 4 additional variables (B-E)
- There are 5 explanatory variables in the full model
Frank Harrell's current rule of thumb about events-per-variable (EPV) in a logit model is that you should have 20 EPV to minimize the risk of overfitting. (Some other authors may still recommend 15 EPV.)
- https://discourse.datamethods.org/t/author-checklist/3407#arrow_rightoverfitting-and-lack-of-model-validation-26
Your full model has 10/5 = 2 EPV. Even a model with A as the only variable has only 10 EPV. So it is not surprising that the CIs are wide. The short answer is that you need a much larger sample in order to achieve the desired 15 or 20 EPV.
PS- David, I do not understand why you said all of the outcomes are the same. As noted above, outcome variable Y=1 for 10 patients/subjects and Y=0 for 30 patients/subjects. That's how I understood it, at least.
Bruce thank you for a good explanation overall. However he says only 10 events occurred. I take that to mean there are 30 nonresponses. Best wishes, David BTW I haven't seen Harrell's current recommendation but what I have seen suggests that those rules of thumb are often bad ideas though I respect Harrell very much. David
So are you assuming Y is missing for the 30 subjects who did not have an event? As I said above, I assume those who did not have an event have Y=0 (no event) rather than Y=1 (event).
Take a look at the unadjusted odds ratio and see if the precision is better? And see if the unadjusted point estimate is any different from the adjusted estimate. Adjustment is intended to reduce bias, but in exchange you are getting high variance. If your unadjusted ratio has lower variance and the ratios don't differ much then your making a bad trade between bias and variance. Just live with a more precise, but somewhat biased estimate.
- Badges
- Science topic
- Similar topics
- Clinical Pharmacology
- Dosing