please see

Fawcett, T. (2006). An introduction to ROC analysis. Pattern recognition letters, 27(8), 861-874.

Article Introduction to ROC analysis

Kubilay Muhammed Sunnetci Thank you for your response. The document suggested by you discusses the ROC curve and how the ROC curves can be utilized for various classification problems. My concern is still not clear from the document. I have drawn the ROC graph by considering the FPR range from 0 to 0.2. The ROC curve seems like the normal ROC curve. But when I calculate the AUC value for the same ROC curve, it shows very less. I have attached the ROC curve which I have plotted for your reference. The AUC value I am getting is 0.33.

Please see if I am on the correct path or not.

Dear Rajesh

Maybe I can explain to you like this: ROC curve cannot be drawn for a single confusion matrix. Here, the ROC curve is plotted for all threshold values. FPR and TPR are plotted from 0 to 1. Why is there no after 0.2, it should be. While the AUC of the random classifier is 0.5, 0.33 can be defined as a failed model. I recommend plotting ROC curves directly using MATLAB or PYTHON.

Kubilay Muhammed Sunnetci Thank you for the details. Let me give you the details of the problem I am working on. I am classifying the malicious and non-malicious customers using k-nearest neighbors. Here I am taking the different k values and evaluating the model performance. So, for different k values, I am getting different confusion matrices. With these confusion matrices, I am plotting the ROC curve and calculating the AUC.

In my problem, my goal is to minimize the number of False-Positives because if any non-malicious customer is detected as a malicious one then the power utility has to go for an on-site inspection that incurs some costs. So, my model should classify the customer as accurately as possible.

Also, when I searched about the FPR range for the ROC curve, I came to know that first, we need to normalize both TPR and FPR values and then plot the ROC curve. Is this way proper to go with?

The false positive rate is essentially a measure of how often a “false alarm” will occur — or, how often an actual negative instance will be classified as positive

A classifier with the perfect performance level shows a combination of two straight lines – from the origin (0.0, 0.0) to the top left corner (0.0, 1.0) and further to the top right corner (1.0, 1.0). A ROC curve represents a classifier with the perfect performance level.

The ROC curve shows a trade-off between TPR and FPR (or false negatives and false positives). It plots TPR vs FPR at different thresholds. If we lower the classification threshold, we will classify more observations as positive, increasing True Positives.

Best way to reduce false positive of binary classification to...

https://www.napier.ai/post/6-ways-to-reduce-false-positives-in-sanction-screening

GOOD LUCK

Dear Rajesh

First of all, the single best k value for k-nn can be determined by the models. Here you can also optimize different evaluation parameters. But I can recommend you: draw the ROC curve of the model on platforms such as MATLAB. The confusion matrix you obtained is calculated for a single threshold value of the classifier. Therefore, you may get erroneous results. While training the classifier, all stages are evaluated and the ROC curve is drawn.

Also, you can use svm, ensemble, LSTM, deep learning architectures as an alternative to k-nn. If you want to create a more successful classifier, you can use the majority voting technique.

At this point, of course, some of the predicted classes may be more important. You can review Misclassification Costs for this. But this changes the confusion matrix. Therefore, it is useful to use up-to-date and new classifiers for more successful models.

[0 1

c 0]

(see https://www.mathworks.com/help/stats/classification-with-unequal-misclassification-costs.html)

It is expected that the proposed model will be able to successfully classify in all classes. For this, metrics such as sensitivity, specificity, precision, f1 score are calculated. Therefore, it can be used in other models instead of just using k-nn.