I don't know exactly what other measures might be used. However, it depends on your application. It would be better if you can give information about your application. For example, in medical applications, where you don't want to miss any potential positive sample, recall is a better option. On the other hand, in most retrieval tasks precision or F-score might be a better solution.

Hi, Normally ROC area alongside precision-recall curve is used to compare the classifier performance. But sometimes for example in multiclass classification ROC curve is not the best solution (given k classes you will need k(k-1) ROC curves). I have seen people reporting overall accuracy in that case. Moreover, the performance of an individual classifier can be reported in the form of a contingency table created using a single best operating point on the ROC curve . 

Seconding everything R. Ahmed stated about contingency tables, and adding another useful summary of the ROC to include in that table can be the area-under-curve metric. (Especially if you don't want to insist on a particular policy of how much worse a false positive is than a false negative.)

Hi everybody

Mehdi, I am working on a classification of textual educational data.   I am a aware that the application plays a role in giving weights to performance measures but I wanted to know more about different measures (other than F and Accuracy) and how it can be formulated. for example one measure which I was told about is the interpretability of the classification model another one is the computational efficiency (learning time and reasoning time)

This article by Sokolova and Lapalme may give you some ideas about the type of measures you need for your classification task

http://www.sciencedirect.com/science/article/pii/S0306457309000259

http://www.sciencedirect.com/science/article/pii/S0306457309000259

http://graphpad.com/quickcalcs/McNemar1.cfm

Really depends on your objective in the study that you are using such algorithms. for example if you aim to use the classifiers in a Web scale analysis, in most of the cases execution time will weigh much higher than even the performance of the classifier (i am talking of millions of instances and hundred thousands of features). I even encountered cases where memory consumption is an important factor.

If your aim is precision then accuracy is a good criteria, though runtime can also be important. Again the nature of the problem is important here. If the model that is to be trained is to be used on an online setting, training time is almost of no consequence. However, if the model is generated successively (i.e. For storage decisions on a search engine after each update) training time is essential.

If you really are comparing different classifiers, statistical analysis is the best method (like a-test, z-test etc, some method that can tell how non-random your classifier's predictions are, though the method again depends on the nature of your experimentation and data, some statistical methods are just not applicable to all datasets)

But in the end the criteria is decided by the objective of the work.