The question (?) that always arises is how it can obtain the “best” (in terms of accuracy); the solution is fusion. It is well known that the “best” fusion solution is one that satisfies the Neyman-Pearson (NP) criterion. The NP criterion characterizes the fusion solution that maximizes the True Accept Rate (TAR) for a fixed value of False Accept Rate (FAR).

In the case of a biometrics verification system, two error rates are evaluated which vary in opposite directions: the false rejection rate FRR (rejection of a legitimate user called “the client”) and the false acceptance rate FAR (acceptance of an impostor). The decision of acceptance or rejection of a person is thus taken by comparing the answer of the system to a threshold (called the decision threshold). The values of FAR and FRR are thus dependent on this threshold which can be chosen so as to reduce the global error of the system. The decision threshold must be adjusted according to the desired characteristics for the application considered. High security applications require a low FAR which has the effect of increasing the FRR, while Low security applications are less demanding in terms of FAR, EER denotes Equal Error Rate and it‟s the point where (FAR=FRR). This threshold must be calculated afresh for each application, to adapt it to the specific population concerned. This is done in general using a small database recorded for this purpose.

Performance capabilities have been traditionally shown in the form of ROC (receiver- or relative-operating characteristic) plots, in which the probability of a false-acceptance is plotted versus the probability of a false-rejection for varying decision thresholds. Unfortunately, with ROC plots, curves corresponding to well-performing systems tend to bunch together near the lower left corner, impeding a clear visualization of competitive systems.

More recently, a variant of an ROC plot, the detection error tradeoff (DET) plot has been used, which plots the same tradeoff using a normal deviate scale. This has the effect of moving the curves away from the lower left corner when performance is high and producing linear curves, making system comparisons easier [33]. Although the complete DET curve is needed to fully describe system error tradeoffs, it is desirable to report performance using a single number. Often the equal-error-rate (EER), the point on the DET curve where the FA rate and FR rate are equal, is used as this single summary number. However, the suitability of any system or techniques for an application must be determined by taking into account the various costs and impacts of the errors and other factors such as implementations and lifetime support costs and end-user acceptance issues.

Hi

·         The ROC curve is a visual characterization of the trade-off between the FAR and the FRR. In general, the matching algorithm performs a decision based on a threshold which determines how close to a template the input needs to be for it to be considered a match. If the threshold is reduced, there will be fewer false non-matches but more false accepts. Conversely, a higher threshold will reduce the FAR but increase the FRR. the compromise point is when the rate at which both acceptance and rejection errors are equal : it is the Equal error rate or crossover error rate (EER or CER):. The value of the EER can be easily obtained from the ROC curve.

Thanks

I guess neither Mohmed nor Aloui got my question. My original question is - As we know ROC is easy to draw when you set a global threshold for all the users in the database, whereas when you set user specific thresholds meaning different threshold for each user, you get different ROCs for different users. What is the best way to to present such ROCs when you have more than 50 users. I hope it make sense now.

Let us be very specific in this discussion of FAR and FRR terminologies. The word imposter is used in both cases and is correct in only ONE instance ie.FRR. When an individual deliberately disguises his/her true identity that individual is an imposter because he/she is in the data base and should be accepted, but is rejected falsely.They,too, are included among those FRRs rejected due to ergonomic factors.

A FAR individual is either someone who is accepted because he/she deliberately poses as someone else ( that individual is an impersonator)  as opposed to someone  accidentally confused with someone else in a data base

Although this response does not address itself to the original question the change in terminologies is long overdue.

Why you would like to have different threshold for each user?

Can you please clarify if you are targeting a tailored solution for each user?What will happen If a new user is added to your system, will you again calculate a new threshold for this new addition?

What kind of classification mechanism you will pick?

In a verification also known as authentication setup (not the identification) you do not add users to the system, you basically develop a model/profile for every user and can have user-specific threshold.

Hi Rajesh,

you have mentioned that you want tradeoffs between the FAR and FRR and I beleive the EER could be a good metric for calaculting the authentication performance as it shows a balance between the FAR and FRR. if you would like to calculate this metric, i am quite happy to share my code with you mate.

Kind Regards.

Rajesh,

Maybe not the best way, but I usually plot the ROC curve with confidence bands derived via bootstrap sampling (or some other sampling method). This requires fixing one error type (I usually fix the FAR), interpolating the other error for each user, then sampling across users. Basically:

I think this gives a rough estimate of variability in performance across users. You'll end up with a graph that looks like this:

https://github.com/vmonaco/pohmm-keystroke/blob/master/figures/example_roc.pdf

For example, see code here:

https://github.com/vmonaco/pohmm-keystroke/blob/master/pohmm_keystroke/plotting.py#L58

Thanks everyone for your inputs.

Vinnie Monaco: I liked your idea. Special thanks for sharing the code. Its really helpful.

Classification Assessment Methods

This paper introduces a detailed explanation with numerical examples many classification assessment methods or classification measures such as:

Accuracy, sensitivity, specificity, ROC curve, Precision-Recall curve, AUC score and many other metrics. In this paper, many details about the ROC curve, PR curve, and Detection Error Trade-off (DET) curve. Moreover, many details about some measures which are suitable for imbalanced data are explained.

Your comments are highly appreciated.

https://ac.els-cdn.com/S2210832718301546/1-s2.0-S2210832718301546-main.pdf?_tid=20353999-8c84-422a-8191-fd1059469272&acdnat=1536185805_03b19e159f0675a01f35512961973eb5

Presentation of the paper is here

https://www.researchgate.net/publication/327403649_Classification_assessment_methods_a_detailed_tutorial