Dear Gautam, in my opinion it is "fully abstract", not practical question. For such situations we could not use these criteria. Moreover, even if TP+FP = 1 or 2 (or TP+FN = 1 or 2) to estimate our solutions by means of standard expressions for precision and sensitivity isn't good idea. Generally speaking, our estimations should be statistically stable (steady, sustained, strong, persistent, ...). For low values of TP+FP (or TP+FN) it is impossible to support this requirement.

Dear Gautam, the reason is following:

Function_of_Average =/= Average_of_Function.

For example, for k=2 and Sensitivity:

Aver_Sens = ( TP_1/(TP_1+FN_1) + TP_2/(TP_2+FN_2) )/2 - it is correct result

Sens_Aver = TP_Aver/(TP_Aver+FN_Aver), where

TP_Aver = ( TP_1 + TP_2 )/2, FN_Aver = ( FN_1 + FN_2 )/2

Regards, Sergey.

Hi,

in the two-class case one just gets the average of the measures computed per each fold/matrix (sometimes errors come from considering the average as an associative op.)

In the multiclass case you may have different measures, as you may opt for micro- or macro-averaging.

A nice reference book on this subject is:

Japkowicz & Shah: Evaluating Learning Algorithms: A Classification Perspective. CUP (2011)

Thanks Sergey Porotsky , It seems correct reason for accuracy difference. The code i used to calculate result is https://stackoverflow.com/questions/54366575/why-accuracy-calculated-by-confusion-matrix-is-different-than-accuracy-returned

Thank you Nicola Fanizzi for suggesting reference book. i really found this book helpful. Regarding micro or macro-averaging as you mentioned in your answer, is it same thing as answer explained by Sergey Porotsky ?

You're welcome. It is related but not the same.

You were considering a binary case, i.e. with a 2 x 2 conf. matrix per fold.

Micro-/macro-averaging is considered with multi-class problems, i.e. with K x K matrices. See

https://datascience.stackexchange.com/questions/15989/micro-average-vs-macro-average-performance-in-a-multiclass-classification-settin/16001

Additionally, note that some indices don't offer good estimates of the performance in case of imbalanced datasets.

Sergey Porotsky Nicola Fanizzi Sir, could you please suggest me what value of precision and sensitivity should be considered when denominator is zero, i.e ( TP+FP =0 and TP+FN=0 respectively). Someone is suggesting precision and sensitivity will be zero as there is no classification while someone suggesting it can be considered as 100% and someone suggesting it could be zero or 100% there is no way to know (ref: https://stackoverflow.com/questions/44008563/zero-denominator-in-roc-and-precision-recall). Even in books it is not mentioned clearly. I am doing multi-class classification and getting denominator zero for 2 to 3 class. Thank you.

Dear Gautam, in my opinion it is "fully abstract", not practical question. For such situations we could not use these criteria. Moreover, even if TP+FP = 1 or 2 (or TP+FN = 1 or 2) to estimate our solutions by means of standard expressions for precision and sensitivity isn't good idea. Generally speaking, our estimations should be statistically stable (steady, sustained, strong, persistent, ...). For low values of TP+FP (or TP+FN) it is impossible to support this requirement.

Hi i assume your are solving a binary classification problem, can try to fix a random seed for your experiment and see if you get the same results. I.e ( from our understand maybe everytime your run the experiment the data is been reshuffled and maybe thats why your getting different result. Please let us know of your findings. It a very good question

HI Burhan Rashid Hussein, i am working on multiclass classification and seed value defined as 'random.seed(42)'. Snapshot of code and result is available on https://stackoverflow.com/questions/54366575/why-accuracy-calculated-by-confusion-matrix-is-different-than-accuracy-returned

Burhan Rashid Hussein, hello! If I have understood question of Gautam well, results don't depend of random seed - it was same experiment both for Sensitivity calculation due to sensitivity of different folds and due to calculation, based FP_Average, FN_Average, etc. May be, I am wrong ?