If you're using a test that has a sensitivity of for instance 90% and specificity of 80% to test for the presence of disease X in a population of 1000 individuals. Let's say 240 individuals have a positive test and 760 individuals have a negative test result, the true prevalence of disease X in this population is given by: (apparent prevalence plus the test specificity minus 1) divided by (the test sensitivity plus test specificity minus 1). In the example above, this works out to be (0.24 + 0.8 - 1) / (0.9 +0.8 -1) = 0.04 / 0.7 = 0.057. Therefore, given a test sensitivity of 90% and a test specificity of 80%, the true prevalence of disease X in this population is 0.057 (5.7%) i.e. 57 individuals are truly diseased but since our test only has a sensitivity of 90% this means that only 90% of the 57 will be correctly diagnosed as being positive (51) while 188 will be false positives. Similarly, 80% of the disease negatives will test negative (754) while 6 individuals are false negatives.
There are many different ways to calculated test characteristics like TP and FP.
But it all depends on your goal and on the kind of data that you are analysing.
A thorough study of the ways to evaluate diagnostic assays is published by Søren Saxmose Nielsen and N. Toft. (Veterinary Microbiology 150 (2011) 115–125)
I think first you need a method which gives you a definite diagnosis of the disease or situation you intended to assess by another method. Then you can easily calculate TP, FP, Specificity, Sensitivity, PPV, and NPV by simple formulas.
For a Bayesian approach to estimating true prevalence from apparent prevalence, you may use our "prevalence" package: http://prevalence.cbra.be/
We also have an online interface to the function for estimating true prevalence from individual samples and a single test: https://cbra.shinyapps.io/truePrev/
In reply to Kim and Mohamad: the Rogan-Gladen estimator for estimating true prevalence is limited in two important ways: 1) it can result in negative estimates, and 2) it does not take uncertainty into account.
More on this in Article Misclassification errors in prevalence estimation: Bayesian ...
For a Bayesian approach to estimating true prevalence, please have a look at our 'prevalence' package: http://prevalence.cbra.be/