I have a multivariate logistic regression model for predicting an event. I then generate 200 bootstrap samples for model validation. I would like to generate a plot of actual probability vs. predicted probability, with ideal, apparent, and bias-corrected plots. All my current code is in SAS.
Dear Nora,
Please find the attached file.
Drawing the confidence level for predicted vs. actual values make it is easy to interpret the simulation results.
Hope in helps.
You can use SCORE statement in PROC LOGISTIC for model validation test. for more info refer to SAS/STAT 9.3 user's guide, page 4111.
http://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#titlepage.htm
Regards
Thank you Ehsan. This table has the estimated as well as the actual vs the estimated probabilities. Will give this a try!
what about https://en.wikipedia.org/wiki/Q%E2%80%93Q_plot, or some of its relatives?
I believe (but I will need to confirm) the plot of interest is the probabilities that are estimated according to the logistic model that I'm proposing, plotted against the probabilities obtained (per subject, etc) according to the bootstrap data models. So I'm plotting the average of all probabilities (over the 200 bootstrap model calculations) vs the estimate obtained from the model applied to my original sample. Still trying to figure out how to obtain a bias-corrected probability which, hopefully, does not deviate from what is apparent.
I've realized there is much more to learn on this topic than learning to produce a graph... I will need to get Frank Harrell's book to learn more about validation and Harrell's optimism. Interesting!
Dear Nora,
Please find the attached file.
Drawing the confidence level for predicted vs. actual values make it is easy to interpret the simulation results.
Hope in helps.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Probability