Dear Rahman,

AUC is a Area Under ROC curve.

1) First make a plot of ROC curve by using confusion matrix.

2) Normalize data, so that X and Y axis should be in unity. Even you can divide data values with maximum value of data.

3) Use Trapezoidal method to calculate AUC. 

4) Maximum value of AUC is one. If you achieve AUC value near to one then your developed method is statistically sound.

For example, AUC>0.9 is good one, AUC

Hello, good morning,

Yes, it's possible. To be more explicit, Based on Theory of probability and Bayes theorem, I made an excel for you where I put an example of confusion matrix and the corresponding ROC curve. (Attachment)

In order to compute AUC, many researchers say that the AUC is related to the Gini coefficient (G1 =  2xAUC - 1). The set of points needed to calculate the equation  sum are in Excel document and de Gini Equation is in

http://en.wikipedia.org/wiki/Gini_coefficient

I know only these two ways to compute the AUC: using trapezoidal approximation or using Gini coefficient.

Furthermore, a interest discussion about the rapidminer AUC compute is in:

http://rapid-i.com/rapidforum/index.php?topic=833.0

Best Regards,

Jorge, from University of Aveiro.

Thanks Jeorge..very helpful..

Dear Gt,

Area under ROC curve (AUC) can be calculated via  trapezoidal approximation. These are mathematical equations and  can be directly calculate in MATLAB via trapz function. 

Ahsan 

But how can i calculate the AUC value from the confusion matrix.??

Dear Rahman,

AUC is a Area Under ROC curve.

1) First make a plot of ROC curve by using confusion matrix.

2) Normalize data, so that X and Y axis should be in unity. Even you can divide data values with maximum value of data.

3) Use Trapezoidal method to calculate AUC. 

4) Maximum value of AUC is one. If you achieve AUC value near to one then your developed method is statistically sound.

For example, AUC>0.9 is good one, AUC

Dear Ahsan,

Your information is really valuable. I am working on multi class classification. I want to check the quality of my classifier for multi class classification using AUC. I am stuck up at how to find AUC for my problem using MATLAB. Moreover I just wanted to clarify as to what is trapezoidal method and how does it help in finding AUC in MATLAB. Does your way work for multiclass classification too??

Thanks in advance.

Dear Manas,

Make a stats for 'True positive rate' and False positive rate. True positive means how many time you detect positive result when actual positive is present. False positive when you detect positive result while actual result is negative. 

After detecting these two rates make a plot which is ROC plot. Now use 'trapz' command to detect AUC. 

Thanks :)

Dear Ahsan,

Thanks for your reply. It was really informative. I am bit confused as to its efficiency with respect to multi class classification problems? Does it produce acceptable results using trapz command in matlab for getting AUC?

One more clarification on using trapz command in matlab. Am I correct in writing trapz(FPR,TPR), where TPR and FPR as vectors ?

Again.....Thanks in advance

Dear Ahsan,

I got the AUC. Now is there any way in matlab to find volume under the surface (VUS) for multi class classification?

Dear Manas and Ahsn,,

this function is probably helpful for computing AUC using matlab  

[~,~,~,_AUC] = perfcurve(Traning_label,Final_prediction,' positive class);

however, this function  need to determine the positive class first either 0 or 1 depend

So, if you have multiclass you can compute the  AUC for each class then take the average to become the average AUC for you system

Hope this will help you

Hi,

By using Perfcurve (in Matlab) you can calculate n different ROC curves for each class. In this case, you consider the current class as the "Positive class" and all other classes as the "Negative class".

Another (more reliable) way is to calculate AUC (from ROC: based on Perfcurve in Matlab) between all classes pairwise and then calculate the unweighted pairwise dissimilarity of classes (Hand and Till, 2001):

AUC_tot=2 *SUM(AUC(ci,cj))/(|C|(|C|-1)).

Hope this helps you

% An example for binary classification

TP =251;

TN=238;

FN=10;

FP=51;

TPR= TP/(TP+FN);

FPR = FP/(FP+TN);

X = [0;TPR;1];

Y = [0;FPR;1];

AUC = trapz(Y,X) % AUC = 0.8926

Or you can just average TPR and TNR

I have been following this discussion for sometime, i have been looking for the same thing. Can we calculate the AUC using the confusion matrix ? I found a paper that calculate AUC from the confusion matrix. The paper is attached and is titled as (PIndroid: A novel Android malware detection

system using ensemble learning methods).

According to the authors,

AUC = 1/2 ((TP/( TP + FN )) + ( TN / ( TN + FP)) ), can anybody confirm this ?

Thanks Mohammad Ahmad for the recommended paper, it was very helpful.

Classification Assessment Methods

This paper introduces a detailed explanation with numerical examples many classification assessment methods or classification measures such as:

Accuracy, sensitivity, specificity, ROC curve, Precision-Recall curve, AUC score and many other metrics. In this paper, many details about the ROC curve, PR curve, and Detection Error Trade-off (DET) curve. Moreover, many details about some measures which are suitable for imbalanced data are explained.

Your comments are highly appreciated.

https://ac.els-cdn.com/S2210832718301546/1-s2.0-S2210832718301546-main.pdf?_tid=20353999-8c84-422a-8191-fd1059469272&acdnat=1536185805_03b19e159f0675a01f35512961973eb5

Presentation of the paper is here

https://www.researchgate.net/publication/327403649_Classification_assessment_methods_a_detailed_tutorial

Henry,

I highly doubt that the ROC's formula in the paper i attached is correct. I have looked into many other publications and none uses the confusion matrix to calculate the ROC the way that paper does. I used WEKA to do some basic classification using my own dataset, and compared the ROC generated by WEKA with the ROC calculated from the confusion matrix as reported in that paper. All the ROCs were different ! Some were not significantly different, but others were indeed different. i wouldn't use that formula.

Dear Mohammad Ahmad

I'm using weka as well, and the reported values of auc do not correspond to the formulas that appear in the paper titled "PIndroid: A novel Android malware detection system using ensemble learning methods" which I agree.

For instance, using Random Forest the value of auc (using breast-cancer.arff) is 0.646 and they should be 0.6108

Confusion matrix

a b

José Daniel López-Cabrera You are right. I checked and compared the ROC value using 10 different ML algorithms, and none matched the result from the formula provided.

In cases of multiple classes , can the AOC be calculated by taking average values of the respective individuals or the higher AOC will be preferred?