If you want a visual representation I agree with Sergey that you can plot each class ROC curve separately.
To get an estimate of the overall classification performance you can use the area under the curve (AUC) for multi-class classification presented in the Hand and Till 2001 paper (doi: 10.1023/A:1010920819831). This metric has been implemented in Python by Stuart Lacy and you can find the code at https://gist.github.com/stulacy/672114792371dc13b247
The 'pROC' R library has also an implementation of this metric ('multiclass.roc' function) as already mentioned by R.V.S. Lalitha. I haven't used MATLAB in a while and I am not aware of any MATLAB implementations.
To the best of my knowledge, ROC curve is defined for binary class classification task and not for multi class classification task.
There are ways t o calculate Sensitivity, Specificity, and MCC for multi class classification task but as far as I know, you can not plot ROC curve for multi class classification task.
Please refer to the following page:
https://en.wikipedia.org/wiki/Receiver_operating_characteristic
To calculate Sensitivity, specificity, and MCC for multi class classification tasks, please refer to the following article:
Proposing a highly accurate protein structural class predictor using segmentation-based features
Use R Graphs
View this url.
http://www.inside-r.org/packages/cran/proc/docs/multiclass.roc
View this url for matlab code.
http://in.mathworks.com/help/stats/perfcurve.html?requestedDomain=www.mathworks.com
I looked in google and thinked myself. There are exists several ways of doing this: 1) declare that any case (in any pixel) when class of pixel in segmentation is not the same as in etalon is an error in classification and any case when class of pixel in segmentation is the same as in etalon is a correct result. It is equal to convertation of your segmentation and etalon in two classes. Draw ROC for this"binary" case; 2) draw ROC for every class on the same graph like in http://scikit-learn.org/stable/auto_examples/model_selection/plot_roc.html 3) draw in Volume (3D).
Hello,
Firstly, you must know that there is no classical definition for Precision and Recall in multi class problems. So, there is no standard method for drawing the ROC curve in the multi class problems. Since, the sum of values in the denominator for calculating the Precision or Recall is zero, it is impossible to calculate them (the result will be NaN in Matlab). Some paper just redefine mentioned metrics by themselves, but they are not standard. For instance, you can see the following paper:
You also can use the mentioned metrics for calculating your own Precision and Recall. Then, you can just draw ROC by your own metric. In my opinion, it is good for your own analysis but for reporting a paper; I don't think so! Besides, you can report the accuracy as well as some supportive metrics or diagrams. I just offer you to see this Matlab example in the following address:
http://www.mathworks.com/help/stats/fitcecoc.html
I wish this information will be useful for you.
Best Regards,
Tony.
http://www.mathworks.com/help/stats/fitcecoc.html
For multi-class classification you need alternative quality depictions. Check this out (specially suited if classes are imbalanced):
http://www.mathworks.com/matlabcentral/fileexchange/30914-entropy-triangle
Hi Padmavathi,
As has been shown above, currently there is no way to plot a ROC curve for multi-class classification problems as it is defined only for binary class classification. However, if you are insisting to do that, I suggest to binaries the problem, in which you can examine your classification methods using one class versus all, and repeating the process till covering all the classes, then you can plot a ROC curve for each class and compare the results. Hope that helps you and you can find more information from the following link
http://sci2s.ugr.es/sites/default/files/files/Teaching/GraduatesCourses/SIGE/v2-Addressing%20Multi-class%20Problems%20by%20Binarization.pdf
Kind regards
Ahmed
If you want a visual representation I agree with Sergey that you can plot each class ROC curve separately.
To get an estimate of the overall classification performance you can use the area under the curve (AUC) for multi-class classification presented in the Hand and Till 2001 paper (doi: 10.1023/A:1010920819831). This metric has been implemented in Python by Stuart Lacy and you can find the code at https://gist.github.com/stulacy/672114792371dc13b247
The 'pROC' R library has also an implementation of this metric ('multiclass.roc' function) as already mentioned by R.V.S. Lalitha. I haven't used MATLAB in a while and I am not aware of any MATLAB implementations.
As in several multi-class problem, the idea is generally to carry out pairwise comparison such as one class vs. all other classes, one class vs. another class. In other hand, you should compare and plot ROC curve for class 1 against classes 2, 3, and etc. Thus in next step, you compare and plot class 2 against classes 1, 3, and etc.
- Badges
- Science topic