I implemented a classification project with naive bayes, svm and multilayer perceptron. for naive bayes, I did that with python and used macro average precision and recall to calculate the f-measure. But for the two other methods, I used weka. Now the question is, does weka use also macro precision and recall or does it use micro ones? (I need to know that because I want to compare the results of these three methods, so they should be same).
Hi,
I think you can mange to use both in Weka package.
This link may help http://www.uky.edu/~nyu222/tutorials/Weka.htm
Micro- and Macro-average of Precision, Recall and F-Score
I posted several articles explaining how precision and recall can be calculated, where F-Score is the equally weighted harmonic mean of them. I was wondering- how to calculate the average precision, recall and harmonic mean of them of a system if the system is applied to several sets of data.
Tricky, but I found this very interesting. There are two methods by which you can get such average statistic of information retrieval and classification.
1. Micro-average Method
In Micro-average method, you sum up the individual true positives, false positives, and false negatives of the system for different sets and the apply them to get the statistics. For example, for a set of data, the system's
True positive (TP1)= 12 False positive (FP1)=9 False negative (FN1)=3
Then precision (P1) and recall (R1) will be 57.14 and 80
and for a different set of data, the system's
True positive (TP2)= 50 False positive (FP2)=23 False negative (FN2)=9
Then precision (P2) and recall (R2) will be 68.49 and 84.75
Now, the average precision and recall of the system using the Micro-average method is
Micro-average of precision = (TP1+TP2)/(TP1+TP2+FP1+FP2) = (12+50)/(12+50+9+23) = 65.96 Micro-average of recall = (TP1+TP2)/(TP1+TP2+FN1+FN2) = (12+50)/(12+50+3+9) = 83.78
The Micro-average F-Score will be simply the harmonic mean of these two figures.
2. Macro-average Method
The method is straight forward. Just take the average of the precision and recall of the system on different sets. For example, the macro-average precision and recall of the system for the given example is
Macro-average precision = (P1+P2)/2 = (57.14+68.49)/2 = 62.82 Macro-average recall = (R1+R2)/2 = (80+84.75)/2 = 82.25
The Macro-average F-Score will be simply the harmonic mean of these two figures.
Suitability Macro-average method can be used when you want to know how the system performs overall across the sets of data. You should not come up with any specific decision with this average.
On the other hand, micro-average can be a useful measure when your dataset varies in size.
Reference:
http://rushdishams.blogspot.in/2011/08/micro-and-macro-average-of-precision.html
Hi,
To correct the upper presented information by the colleagues (appreciate their valuable contribution), WEKA does both automatically without using the APIs as well as without having any packages.
In WEKA's Explorer, the performance statistics are calculated by micro-averaging across test sets. In the Explorer, the values in the row labeled "Weighted Avg.” are calculated as a weighted arithmetic average of the per-class results in the corresponding columns (i.e., as a weighted average of the micro averaged per-class results). The weight of each class in this average is given by the prevalence of the class in the pooled test sets. However, the Experimenter generates the micro and macro averages.
HTH.
Samer, PhD.
Samer Sarsam
please, how to find micro average result in weka tool ?
for one dataset has tow classes 0,1 as my figure attached
Is the weighted average f-measure the same as micro averaged f measure?
Yasmine Ali
For a multi-class confusion matrix (including two-class situations like yours), micro-averaged precision, recall, and F-measure are all the same and identical to classification accuracy as measured by the percentage of correctly classified instances (not shown in the output you attached, but obviously also output by WEKA).
HTH.
Dr. Samer Sarsam
How to open arabic character excel file in weka ?
after save excel file encoding utf-8 then convert it to csv then open in weka tool , my arabic character become error symbols in weka>what is the solution?
- Badges
- Science topic
- Similar topics
- Filing