You can only compute something if you have a plausible statistical model of your authentication method. 

Once you have that the false positive rate becomes the chance that you accept an authentication despite not having the authorisation data, and the false negative that you reject an authentication despite having the authorisation data. 

The terminology "negative" and "positive" here, is based on the null hypothesis that you are not authorised.  The null hypothesis is always the conservative usually boring hypothesis.  Therefore if the test gives a positive it means you reject the hypothesis. E.g. in medical testing the null hypothesis H0 is that a patient does not have cancer. If the test is positive i.e. detects some feature like a specific protein, then you accept the alternative hypothesis that the patient does have cancer. A false positive rejects the null hypothesis and flags the patient as having cancer despite being healthy.  Likewise a false negative does not flags the patient as having cancer despite being ill.

First, you have to devise an emperical experiment to evaluate how your method works with probands. Afterwards, you can invoke statistical tools (like SPSS) to forecast results for higher sample sizes.

I have some experience testing biometric authentication products. The risk is always that one jumps too early into ad hoc testing before an experiment design is made. So I agree with the previous answers. make sure the use case is clear and that there are acceptance/success criteria *before* you start testing.

Next to that: the terms to use for authentication are False Accept and False Reject. Those give the False accept rate (FAR) and the False reject rate (FRR). But they are of course the same as FP and FN.

Normally authentication methods that need to be tested, like biometric systems, produce a score for every authentication attempt. (trust score). One of the aspects of the biometric system is that you set a threshold depending on the functional requirements. But that is the actual appication. To measure the performance of an authenticator regardless of the trhreshold setting, you need to create the FAR and FRR curves. Or better the Detection error trade-off curve (DET). 

There is no tool except for R, SPSS and python :-)

About significance (and how many tests you need) is discussed a bit in "Common Criteria - Common Methodology for Information Technology Security Evaluation - Biometric Evaluation Methodology Supplement [BEM]" from British government ( http://www.cesg.gov.uk, site has maintenance issues at the moment of writing)

good luck

Weka tool is used to calculate FP and FN. It is depend on your system. We also can implement your application for the same. For this, we can find the formulae for the same in many literature paper.

D P Gaikwad

I assume you are looking at binary classification system

receiver operating characteristic (ROC) curves that plot the true positive rate (sensitivity) on the vertical axis against the false positive rate (1 - specificity, fall-out) on the horizontal axis for all possible threshold value would be a good choice.

ROCR package in R can do that.

I have a follow up question which is relevant once you have measurements.

If you have done some tests and you have n authentication attempts with scores and labels (fraud/non-fraud), then you can create an ROC curve (or DET curve depending on your preference). However with a small n the DET curve could very well be very different.

I have added some examples (see attachment) of a (simulated) authenticator which gives a confidence score, typically a biometric authenticator. figure A, 1 test run with n=600 (fraud/non fraud about 1;1) will vary a lot when repeated, purely due to randomness (figure B). If n=4000, then the variability is visibly less (figure C).

My question: how to get the correct information on the performance of the authenticator when you have only one test run with relatively small number of authentication scores (n)?

For small data I would suggest to look at Bayesian Approach. It updates the belief based on the new data.  Further you can mix your subjective beliefs.