I have 50 abnormal and 50 normal data set. I have set the classification process with 2 classifiers (LR and NB) and 10-cross validation.
The tool gave a confusion matrix and performance metrics. The number of instances failed in LR is 8 and that in NB 15.
If the FN and FP is 2 and 6 (using confusion matrix), then how do check if the failures have been incorrectly classified when they are in
(a) only when the validation set
(b) whether they are in validation or training set
(c) only when they are training set
Weka tool gives average performance and hence the query.
Thanks in advance for your time.
I think that the confusion matrix shows the result of the cross validation, which means that the errors you see are when the samples are in the test set.
In any case, if you want to get the results of specific instances then in the Explorer go to "More options" and add the "Output predictions" option,
In cross-validation data randomly separates to two subsets; training and test set. Using a 10-fold cross-validation, the learning and then testing processes take place 10 times, each time with a different test set. The 10 test sets are mutually exclusive and the union is the whole dataset. Each sample will be once and just once in test set, and it will surely used to test the learned machine. So the answer to your question is that the check for the performance takes place when a sample is in the test set (validation set).
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software