Dear Danny,

are you sure that the area under the ROC curve is 0. For me that does not make any sense, as non-informative (pure random) model achieves an area of 0.5? I would propose that you re-check those computations.

Best

Matthias

A random classifier for a balanced set will give 0.5 AUC.

An area of 0 dos not make sense, unless you have an unbalanced dataset (and true probability of one class is 0.9+), and your classifier is putting everything to the highly improbable class.

In that case, simply reversing your classifier prediction should result in AUC~1

Dear Danny,

on how many observations the model was built?

Ad.: point 1 and 2 provide mostly close to univocal information, which suggests to ask in ad. to point 3: do you really need this type of model? Accuracy of 64% is not that bad, but somehow does it really match the r2=~0.08 (r=0.28). 64% accuracy would suggest that the model describes more that 8% of data it was built on.

If my understanding of your description is right, there should be some error during calculations somewhere.

Point 4: For accuracy 64% ROC cannot be 0.--> how many data samples were used to build ROC curve? --> how many observations were used to build and validate the model?

Regards,

Richard Gomolka

Danny it seems that there is some problem in this analysis, instead.

You may be interested in using the ADAPT curve:

Lee WC, Wu YC. Characterizing decision-analysis performances of risk prediction models using ADAPT curves. Medicine 2016;95(2):e2477.