I guess you already sorted your output according to the score given by your classifier to the positive class. Then you have to plot the pairs TPR, FPR you got. If you have few point it will look like more like a scalator than a curve.

You can do the sorting and computations in Excel. In these slides you can find an example:

http://www.cs.uic.edu/~liub/teach/cs583-fall-06/CS583-supervised-learning.ppt

The book is highly recommended.

I guess you already sorted your output according to the score given by your classifier to the positive class. Then you have to plot the pairs TPR, FPR you got. If you have few point it will look like more like a scalator than a curve.

You can do the sorting and computations in Excel. In these slides you can find an example:

http://www.cs.uic.edu/~liub/teach/cs583-fall-06/CS583-supervised-learning.ppt

The book is highly recommended.

Dear friend

Greetings.

The contingency table can derive several evaluation "metrics" (see infobox). To draw a ROC curve, only the true positive rate (TPR) and false positive rate (FPR) are needed (as functions of some classifier parameter). The TPR defines how many correct positive results occur among all positive samples available during the test. FPR, on the other hand, defines how many incorrect positive results occur among all negative samples available during the test.

A ROC space is defined by FPR and TPR as x and y axes respectively, which depicts relative trade-offs between true positive (benefits) and false positive (costs). Since TPR is equivalent to sensitivity and FPR is equal to 1 − specificity, the ROC graph is sometimes called the sensitivity vs (1 − specificity) plot. Each prediction result or instance of a confusion matrix represents one point in the ROC space.

The best possible prediction method would yield a point in the upper left corner or coordinate (0,1) of the ROC space, representing 100% sensitivity (no false negatives) and 100% specificity (no false positives). The (0,1) point is also called a perfect classification. A completely random guess would give a point along a diagonal line (the so-called line of no-discrimination) from the left bottom to the top right corners (regardless of the positive and negative base rates). An intuitive example of random guessing is a decision by flipping coins (heads or tails). As the size of the sample increases, a random classifier's ROC point migrates towards (0.5,0.5).

The diagonal divides the ROC space. Points above the diagonal represent good classification results (better than random), points below the line poor results (worse than random). Note that the output of a consistently poor predictor could simply be inverted to obtain a good predictor.

I forgot to mention that ROC curves are for binary classification. Otherwise it turns out to be more tricky.

In this article you can find clear description (with easy-to-get graphical representation) of the ROC curve creation process  - https://shapeofdata.wordpress.com/2015/01/05/precision-recall-aucs-and-rocs/

Actually you can use the WEKA jar as an external library if you wish to use some of its implementation together with yours. For an example of it you can look at the code associated with the paper:

https://www.researchgate.net/publication/215739042_Fish_recruitment_prediction_using_robust_supervised_classification_methods

All the best

Article Fish recruitment prediction, using robust supervised classif...

try to use t the f-measure which is a combination of tpr and fpr. If you plot the f-measure versus your threshold then you can easily see the big picture.

Here's a simple guide for creating ROCs for a binary classification problem (and the relationship to the quantities such as the raw detection probabilities and TP, FP, TN, FN), as well as some sample 

http://docs.eyesopen.com/toolkits/cookbook/python/plotting/roc.html

One thing to note - these curves should be step functions for discrete data.

http://docs.eyesopen.com/toolkits/cookbook/python/plotting/roc.html

Thanks guys, with your helpful comments I finally made it. I provided a Java code to generate ROC coordinates and easily draw the curve using Matlab Plot.

Hello there,

As you know, ROC curve is a two-dimensional depiction of the accuracy of a signal detector, as it arises on a given set of testing data. Two dimensions are required to show the whole story of how the true positive rate decreases as the false positive rate increases. So, use TP and FP for drawing a two-dimensional curve.

Hi Pooia,

Did you create your own Java code, or did you use a Java package available online to get the coordinates?

I tried to find some online but none of them coped with the way I needed them to work. So I created from the scratch while testing the model on the dataset instances. Actually I integrated the code into my result function while testing the model, and generated my (x, y)s. other packages like Orange or Weka create ROC curves but not as flexible as your code.

Determine TP, TN, FP, FN  for every threshold and calc for each the tpr = TP/(TP+FN) and fpr = FP/(FP+TN). Plot hem against each other, fpr on the x-axis. Use AUC = area under the curve to get a general performance estimate  (by parametric fit or non-parametric estimate via the trapezoid rule or Mann Whitney U-statistic). There are different ways to get a standard error estimate. We used the SE approximation according to the Hanley-MacNeil paper (1982) but there are better alternatives.

You can use matlab.

Today I got to know that this question has caught a lot of attention during the last three years.

This encouraged me to provide some literature that I think is good to clear some doubts:

https://www.sciencedirect.com/science/article/pii/S016786550500303X

It does make much sense to add TP (True Positive) and FP (False Positive) to ROC. TP and FP from a Confusion matrix are mainly for the accuracy of a predictive model, which are subjective measures depending upon on the score cut. On the other hand, ROC along with AUC is not subjective. It measures the predictive power of the model.

I have tried this by installing a plugin 'JFreeChart Renderer'. Multiple ROC curves of different algorithms can be plotted in a graph. It is easy to compare. However, I could not find the configuration to choose or set the different colors for different algorithms, thus it would be able to compare algorithms along with different datasets. Problem here is multiple graphs of different algorithms in different datasets are different.

https://in.mathworks.com/help/stats/perfcurve.html

https://in.mathworks.com/matlabcentral/answers/310548-plotting-roc-curve-from-confusion-matrix

You can get the code to draw ROC from weka.jar file which include all .jar file thats used in weka.

dwnld the jar file