How can I plot/determine ROC/AUC for SVM?

More Sheema Khattak's questions See All

How to interpret the results of PSNR and MSE for the quality evaluation of images?

How can we determine from the results of PSNR and MSE that how is the quality of the images for quantitative and qualitative analysis.

31 December 2017 8,606 3 View

What are the methods used for quantitative/Qualitative analysis in medical/digital image processing?

Methods like ROC/AUC, precision,specificity and accuracy etc are used for quantitative analysis. What are other methods to evaluate the results.

10 November 2017 4,432 5 View

Can Random Walker be compared to Support vector machine?

Since Random walker is used for both segmentation as well as classification. But it works on supervised learning so is it better to comapre it with other segmentation techniques or classification...

03 April 2017 3,870 3 View

Can someone guide me about the interpretation of Relevance Vector Machine results in matlab? And how can i plot a classifying output from the result ?

How to understand what RVM is showing and how to plot an output out from the results?

09 October 2016 2,343 0 View

How to write an introduction in a computer vision research article?

How should be the introduction portion written in a research article to get the paper published in computer vision.

08 September 2016 2,574 2 View

How to write experimental results in scientific way?

How can beginners write research article in such a way that it gets published in a reputable journal especially the crucial portions like experimental results and hence convincing the readers...

07 August 2016 981 8 View

What are the current challenges or under research topics in medical image segmentation?

In order to find abnormalities and eventually disease detection.

06 July 2016 2,090 2 View

What is meant by Random Walker algorithm in the context of image processing?

Also kindly illustrate the key terms in the context of its implementation seeds,masks,labels and beta? what are these terms and what does they represent?

03 April 2016 4,566 2 View

What should be the xlabel and ylabel for svm plot?

SVM Classification results in the form of 2-d plot

08 September 2015 538 1 View

Any new ideas about T.B disease detection in lungs using image processing/computer vision?

For PhD research any new ideas and methods

07 August 2015 8,813 1 View

How to compare two groups with only two measurements?

Hiiiii everyone! I have an enquiry on statistical analysis. I was looking for many forum and it's still cannot solve my problem. I want to compare means of two groups of data but only with two...

03 March 2021 8,796 3 View

What is the most Operating System (OS) which can be INSTALL on Tiny hard devices like Raspberry Pi (RPi) hardware ?

I'm targeting to deploy a mesh network and manually configure MANET routing protocols. I'm preparing scenarios, architectures, and hard devices needed to do that. Are there some step-by-step...

03 March 2021 1,931 5 View

Why should we choose deep learning to machine learning? Why is CNN better for image classification? Can I get Typical Answer?

What Characteristics makes CNN work better?

03 March 2021 1,458 4 View

What are some of the research gaps in machine learning and artificial intelligence in africa?

i would to know some of the research gaps in the artificial intelligence field in most african countries.

03 March 2021 6,145 3 View

What's the best way to measure growth rates in House sparrow chicks from day 2 to day 10?

What's the best way to measure growth rates in House sparrow chicks from day 2 to day 10? Since, the growth curve from day 2 to 10 won't be like the "Logistic curve" it might not follow logistic...

03 March 2021 1,401 3 View

How to increase a model accuracy ?

dear community, my model is based feature extraction from non stationary signals using discrete Wavelet Transform and then using statistical features then machine learning classifiers in order to...

03 March 2021 6,994 5 View

Can NFL theorem be valid in infinite search space in ML?

NFL theorem is valid for algorithms training in fixed training set. However, the general characteristic of algorithms in expanded or open dataset has not been proved yet. Could you show your...

01 March 2021 1,189 3 View

Timeout error!! Processor in the loop (PIL) using STM32F4 Matlab 2018a?

Dear Researchers I am trying to perform a PIL simulation using STM32F4 Discovery board and comunication serial USB TO TTL. During simulation I receive the following timeout error: An error...

01 March 2021 2,327 1 View

What is the difference between L1 and L2 regularization in machine learning?

L1 and L2 regularization

28 February 2021 4,187 3 View

How to eliminate Steady state error from Reinforcement learning genetic algorithm code?

The following code (see 1st 2 images attached) is used to produce PID controller values that are designed to control the system (G). The code finds the PID controller values (noted as k) by using...

28 February 2021 6,560 14 View

Frerk Saxen Popular answer

Split your dataset into a training set and a testing set

Train your SVM using the training set only

Evaluate the testing set using the trained svm model

Compare the pre-known labels L of your testing set with the output of the svm O (step 3)

We want the label matching the output. But we distinguish further between true positive (TP) and true negative (TN). Also, if the label does not match the output, we distinguish further between false positive (FP) and false negative (FN).

You start with TP = TN = FP = FN = 0. Now you go through your testing set and increment TP, if your label matches and is positive. You increment TN if your label matches and is negative, a.s.o.

At the end you have a value for TP, TN, FP, and FN (the so called confusion matrix). This is only one single value in the ROC plot. Look at http://en.wikipedia.org/wiki/Receiver_operating_characteristic to see how to calculate the TPR and FPR for the ROC.

But the svm output is calculated by y = sgn(w^T - b). If you vary the bias b (which is the offset of the hyperplane) and redo step 3 to 7 you get more and more points on the ROC plot

This is quite inefficient because you have to reevaluate the svm model each time. And who is gonna tell you the correct value for b?

If you evaluate your testing set with the svm, simply don't use the signum but instead evaluate y = w^T * x - b (most svm implementation provide only this floating point value anyhow). Now you need to threshold this value to receive a label. And each possible threshold will provide you with a single point in the ROC plot. And who tells you the threshold? Just use every single value your svm produced.

Instead of manually splitting the dataset into a training and a testing set you might want to consider k-fold crossvalidation: http://en.wikipedia.org/wiki/Cross-validation_(statistics)

Costanzo Di Maria

Dear Sheema,

from your question there is not enough detail to know what is that you would like to do.

I assume that your problem is that SVM is a binary classifier which return 0 or 1, and you cannot directly use this kind of output to compute your ROC. If this is the case, one solution can be to take the distance of each of your point to the hyperplane and transform this into a predicted probability by using a binary logistic model. The you can use the predicted probabilities returned by this model to do ROC analysis. I attach an article by Platt for reference. Hope this helps. Costanzo

Frerk Saxen

Amaury Lendasse

You can look in the Matlab LSSVM toolbox how they do

Mervyn Thomas

Costanzo is correct. You will find software to do this in R (which is open source), in particular the kernlabs package by Alexandros Karatzoglou, Alex Smola and Kurt Hornik. See http://cran.r-project.org/web/packages/kernlab/vignettes/kernlab.pdf. This software is very high quality, Smola is a leading researcher in maximum margin methods.

Sheema Khattak

Thank you all. Mr.Costanzo my problem is binary and need to compute ROC for svm applied I am not sure that what should I use svm accuracy or error rate to compute ROC. Mr.Frerek I have done ROC computation on each step /iteration but ROC plot is very non-obvious resembling a stair case ascended.

ROC curve analysis does not use accuracy or error rate. An ROC curve plots sensitivity (y axis) versus 1-specificity (x axis). You have one point for each value that you set as the threshold on your measurement. Your measurement could be the predicted probabilities if you use the approach mentioned above. As Thomas explained above, there are free packages that can help you to do this in R. Or you can use other software like SPSS if you are more familiar with it. I would suggest you do some more reading about ROC curve analysis before going any further. Costanzo

it seems that you are more than half way through. As Constanzo pointed out. ROC plots the false positive rate vs. true positive rate, or sensitivity vs. 1-specificity (which is the same). This is clearly mentioned in the link: http://en.wikipedia.org/wiki/Receiver_operating_characteristic

I recommend you to read this since you might want to interpret your classification results afterwards. A good knowledge about the different performance measurements is crucial.

Because various suggestions have been made according to implementations using R or Matlab, could you tell us which platform you are using?

Thank you. Its Matlab.

Sam Darvishi

Hi Sheema,

As it is a fairly old thread, I assume you've already solved the problem. However, if not, you might find it useful to have look at "perfcurve" documntation in Matlab. Hope it helps.

Thank you. Yes it is old one but I still cudn't solve it, as I have older version of matlab 2013. And the perfcurve is in matlab 2014 which I cudnt get till now.

Hi Sheema please find the attached file. Copy and paste the contents into a .m file in Matlab and save it as p.e.r.f.c.u.r.v.e.m. Then you can use it. Hope it helps.

Thank you

Can you guide me how to adapt this perfcurve for analysis of segmentation method performed on image qualitatively /quantitatively?

I don't totally understand your question, but I suppose having a look at the following links may be of some assistance:

http://au.mathworks.com/matlabcentral/newsreader/view_thread/280317

http://au.mathworks.com/matlabcentral/answers/68138-how-to-do-a-roc-analysis-using-matlab-build-in-svm-not-libsvm

http://stackoverflow.com/questions/20600568/roc-curve-for-a-binary-classifier-in-matlab

interesting!