Hi Marcella,

Diagnosticians use a range of techniques to answer similar questions (although they use specificity in place of selectivity).  This paper by Brad Biggerstaff might be useful. I'm happy to discuss your questions further  - the topic of model accuracy in different contexts is of general interest to me:

http://www.stat.colostate.edu/~bradb/papers/lrgraphfinal.pdf

Yours,

N

Hi,

TP/(TP+FP) is the positive prediction value, at least in literature regarding diagnostic accuracy (e.g. Pepe, Zhou, STARD Statement and others).

You should considerfollowing pairs of estimates

{sensitivity, specificity} = {TP/(TP+FN), TN/(TN+FP)}

or

{TPF, FPF} = {sensitivity, 1-specificity} = {TP/(TP+FN), FP/(TN+FP)}

or

{positive predictive value, negatrive predictive value} = {PPV, NPV} = {TP/(TP+FP), [TN/(TN+FN)}

or

(positive diagnostic likelihood ratio, negative diagnostic likelihood ratio}={DLR+, DLR-}={sens/(1-spec), (1-sens)/spec}

For the graph I recommend sens vs. (1-spec) or TPF vs. FPF, respectively, which is the Receiver operating characteristics (ROC) curve.

For comparisons, it depends whether you have paired measurements or unpaired measurements.

In case of paired observations (all test were applied at the same population) I propose to compare sensitivities via McNemar-Test after choosing the cut-offs of each test in a way, that all tests have the same specificity.

Another way ist to use generalized linear models with log-link (not logit!) and to estimate ratios of TPF = rTPF, again using tests with cut-offs leading to identical specificity.

Kind regards

Thomas

ACOMED Statistik, Leipzig, Germany