How can I plot/determine ROC/AUC for SVM?

More Sheema Khattak's questions See All

How to interpret the results of PSNR and MSE for the quality evaluation of images?

How can we determine from the results of PSNR and MSE that how is the quality of the images for quantitative and qualitative analysis.

31 December 2017 8,749 3 View

What are the methods used for quantitative/Qualitative analysis in medical/digital image processing?

Methods like ROC/AUC, precision,specificity and accuracy etc are used for quantitative analysis. What are other methods to evaluate the results.

10 November 2017 4,527 5 View

Can Random Walker be compared to Support vector machine?

Since Random walker is used for both segmentation as well as classification. But it works on supervised learning so is it better to comapre it with other segmentation techniques or classification...

03 April 2017 3,951 3 View

Can someone guide me about the interpretation of Relevance Vector Machine results in matlab? And how can i plot a classifying output from the result ?

How to understand what RVM is showing and how to plot an output out from the results?

09 October 2016 2,431 0 View

How to write an introduction in a computer vision research article?

How should be the introduction portion written in a research article to get the paper published in computer vision.

08 September 2016 2,656 2 View

How to write experimental results in scientific way?

How can beginners write research article in such a way that it gets published in a reputable journal especially the crucial portions like experimental results and hence convincing the readers...

07 August 2016 1,093 8 View

What are the current challenges or under research topics in medical image segmentation?

In order to find abnormalities and eventually disease detection.

06 July 2016 2,263 2 View

What is meant by Random Walker algorithm in the context of image processing?

Also kindly illustrate the key terms in the context of its implementation seeds,masks,labels and beta? what are these terms and what does they represent?

03 April 2016 4,654 2 View

What should be the xlabel and ylabel for svm plot?

SVM Classification results in the form of 2-d plot

08 September 2015 640 1 View

Any new ideas about T.B disease detection in lungs using image processing/computer vision?

For PhD research any new ideas and methods

07 August 2015 8,885 1 View

How to learn more about SPSS and its Application?

I would like to learn more about SPSS and Its application especially in regards to data analysis. Please suggest me how I can learn more about it. Thank you so much.

11 August 2024 9,101 4 View

Has anyone applied Python in the field of textile engineering for data analysis, automation, or smart textiles?

I'm currently exploring the application of Python in textile engineering, specifically in areas like data analysis, process automation, and the development of smart textiles. I'm interested in...

10 August 2024 7,429 2 View

How can i do multivariate Time Series forecast using MLP, ANFIS and LSTM?

I need the python code to forecast what crop production will be in the next decade considering climate and crop production variables as seen in the attached.csv file.

05 August 2024 2,977 3 View

Need help with my research project on open source SIEM and machine learning?

Hello everyone, I am currently working on a research project that aims to integrate machine learning techniques into an open source SIEM tool to automate the creation of security use cases from...

04 August 2024 3,196 2 View

What are the limitations and challenges of using machine learning for predicting concrete compressive strength in practical applications?

Machine learning (ML) has shown great potential in predicting the compressive strength of concrete, an important property for structural engineering. However, its practical application comes with...

03 August 2024 2,546 2 View

How to choose the journal?

Hello I want a suitable journal in the field of remote sensing and machine learning to be judged quickly. Thank you for your guidance Thanks

01 August 2024 1,799 4 View

Posthoc test lettering in JAMOVI?

Does anyone know of a module for the JAMOVI software that is capable of generating mean separations using the classic letters based on post hoc results (e.g., Tukey test)? If, as I believe, such...

31 July 2024 3,333 4 View

How to back transform the results generated from analyses using log transformed with In(X+1) data?

I am conducting my analysis using SPSS. I log transformed my data using In(X+1) as my data contain zero values. However, when I want to back transform the regression coefficients generated from my...

31 July 2024 7,860 3 View

A Question about Phd thesis?

Hello everyone What is your opinion about the introduction of an expert decision support system in which the rules are extracted from existing data without human intervention, instead of being...

31 July 2024 5,785 4 View

Help on understanding the implementation of Mori Tanaka method on MATLAB?

I am new to Micromechanics and having similar problem with understanding the implementation of the formula's. I would appreciate if anyone can guide me on how to go about getting a scalar value...

30 July 2024 969 0 View

Frerk Saxen Popular answer

Split your dataset into a training set and a testing set

Train your SVM using the training set only

Evaluate the testing set using the trained svm model

Compare the pre-known labels L of your testing set with the output of the svm O (step 3)

We want the label matching the output. But we distinguish further between true positive (TP) and true negative (TN). Also, if the label does not match the output, we distinguish further between false positive (FP) and false negative (FN).

You start with TP = TN = FP = FN = 0. Now you go through your testing set and increment TP, if your label matches and is positive. You increment TN if your label matches and is negative, a.s.o.

At the end you have a value for TP, TN, FP, and FN (the so called confusion matrix). This is only one single value in the ROC plot. Look at http://en.wikipedia.org/wiki/Receiver_operating_characteristic to see how to calculate the TPR and FPR for the ROC.

But the svm output is calculated by y = sgn(w^T - b). If you vary the bias b (which is the offset of the hyperplane) and redo step 3 to 7 you get more and more points on the ROC plot

This is quite inefficient because you have to reevaluate the svm model each time. And who is gonna tell you the correct value for b?

If you evaluate your testing set with the svm, simply don't use the signum but instead evaluate y = w^T * x - b (most svm implementation provide only this floating point value anyhow). Now you need to threshold this value to receive a label. And each possible threshold will provide you with a single point in the ROC plot. And who tells you the threshold? Just use every single value your svm produced.

Instead of manually splitting the dataset into a training and a testing set you might want to consider k-fold crossvalidation: http://en.wikipedia.org/wiki/Cross-validation_(statistics)

Costanzo Di Maria

Dear Sheema,

from your question there is not enough detail to know what is that you would like to do.

I assume that your problem is that SVM is a binary classifier which return 0 or 1, and you cannot directly use this kind of output to compute your ROC. If this is the case, one solution can be to take the distance of each of your point to the hyperplane and transform this into a predicted probability by using a binary logistic model. The you can use the predicted probabilities returned by this model to do ROC analysis. I attach an article by Platt for reference. Hope this helps. Costanzo

Frerk Saxen

Amaury Lendasse

You can look in the Matlab LSSVM toolbox how they do

Mervyn Thomas

Costanzo is correct. You will find software to do this in R (which is open source), in particular the kernlabs package by Alexandros Karatzoglou, Alex Smola and Kurt Hornik. See http://cran.r-project.org/web/packages/kernlab/vignettes/kernlab.pdf. This software is very high quality, Smola is a leading researcher in maximum margin methods.

Sheema Khattak

Thank you all. Mr.Costanzo my problem is binary and need to compute ROC for svm applied I am not sure that what should I use svm accuracy or error rate to compute ROC. Mr.Frerek I have done ROC computation on each step /iteration but ROC plot is very non-obvious resembling a stair case ascended.

ROC curve analysis does not use accuracy or error rate. An ROC curve plots sensitivity (y axis) versus 1-specificity (x axis). You have one point for each value that you set as the threshold on your measurement. Your measurement could be the predicted probabilities if you use the approach mentioned above. As Thomas explained above, there are free packages that can help you to do this in R. Or you can use other software like SPSS if you are more familiar with it. I would suggest you do some more reading about ROC curve analysis before going any further. Costanzo

it seems that you are more than half way through. As Constanzo pointed out. ROC plots the false positive rate vs. true positive rate, or sensitivity vs. 1-specificity (which is the same). This is clearly mentioned in the link: http://en.wikipedia.org/wiki/Receiver_operating_characteristic

I recommend you to read this since you might want to interpret your classification results afterwards. A good knowledge about the different performance measurements is crucial.

Because various suggestions have been made according to implementations using R or Matlab, could you tell us which platform you are using?

Thank you. Its Matlab.

Sam Darvishi

Hi Sheema,

As it is a fairly old thread, I assume you've already solved the problem. However, if not, you might find it useful to have look at "perfcurve" documntation in Matlab. Hope it helps.

Thank you. Yes it is old one but I still cudn't solve it, as I have older version of matlab 2013. And the perfcurve is in matlab 2014 which I cudnt get till now.

Hi Sheema please find the attached file. Copy and paste the contents into a .m file in Matlab and save it as p.e.r.f.c.u.r.v.e.m. Then you can use it. Hope it helps.

Thank you

Can you guide me how to adapt this perfcurve for analysis of segmentation method performed on image qualitatively /quantitatively?

I don't totally understand your question, but I suppose having a look at the following links may be of some assistance:

http://au.mathworks.com/matlabcentral/newsreader/view_thread/280317

http://au.mathworks.com/matlabcentral/answers/68138-how-to-do-a-roc-analysis-using-matlab-build-in-svm-not-libsvm

http://stackoverflow.com/questions/20600568/roc-curve-for-a-binary-classifier-in-matlab

interesting!