Vahideh,

Pushing the class to be the first attribute (after applying "StringToWordVector" filter) is a normal behavior to this filter, and it's not a problem at all. In addition, building the model on the training set is not a good idea. Thus, you need to use the option "Percentage split" and save the resulting model from this option. On the other hand, the error message should be 19 != 2 (not 1 != 2 ) that occurs because your training set and test set are not compatible, which is also an expected issue. After applying StringToWordVector, the attributes in the training set file will be converted into a numeric type, while those in the test set are from the string type. Therefore, you have to convert the test set attributes into a similar type of the training set. Here is the result I had:

=== Summary ===

Correctly Classified Instances 4 100 %

Incorrectly Classified Instances 0 0 %

Kappa statistic 1

Mean absolute error 0

Root mean squared error 0

Relative absolute error 0 %

Root relative squared error 0 %

Total Number of Instances 4

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 full

1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 fill

Weighted Avg. 1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000

=== Confusion Matrix ===

a b

https://stackoverflow.com/questions/41351804/class-index-differ-error-in-weka

Sobhan Sarkar Samer Sarsam

when load model that i created by cross validation and call my arff file this error is present..

attached file is arff and model

please help?

Vahideh Torabi please help for my question

Yasmine Ali

Well, major issue is that the structure of the incoming data does not match with that of the trained classifier. Precisely, the classifier model, that was built based on the training set, has different assumptions about your loaded test set. Those assumptions are NOT matched with the actual test set structure. For instance, WEKA considers the class that can be handled by the classifier model differently from the class of the test set.

Advice: make sure that the test set doesn't processed by "StringToWordVector". StringToWordVector needs to be applied only through the classifier model (i.e., applied on the training set).

Cheers,

Dr. Samer Sarsam

Hello, I am using SMO classifier.

For the training set it works well:

Correctly Classified Instances 5469 91.6234 %

Incorrectly Classified Instances 500 8.3766 %

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

0.949 0.116 0.890 0.949 0.919 0.834 0.916 0.870 walking

0.884 0.051 0.946 0.884 0.914 0.834 0.916 0.894 sitting

Weighted Avg. 0.916 0.084 0.918 0.916 0.916 0.834 0.916 0.882

=== Confusion Matrix ===

a b

Ziad Salem , could you highlight the exact problem in your result?

Dr. Samer Sarsam

Hi,

I wondering about the ?

0.948 ? 1.000 0.948 0.973 ? ? 1.000 walking

? 0.052 0.000 ? ? ? ? ? sitting

Weighted Avg. 0.948 ? 1.000 0.948 0.973 ? ? 1.000

Also the Matrix for b it is 0

=== Confusion Matrix ===

a b

Because no instances are assigned to the class during the learnining process.

Cheers,

Dr. Samer Sarsam

Is this normal or it is a problem? if it is a problem how can we handel it?

You see the training accuracy is 91.62 % while the test accuracy is 94.75% which is a bit strange.

This is normal. However, it is good to have your data well-prepared before applying any algorithm.

I am not sure what you mean by saying "You see the training accuracy is 91.62 % while the test accuracy is 94.75% which is a bit strange", but if 91.62 % was produced within the "Use training set" mode, then it is not really a reliable result. Actually, 94.75% is not strange because prediction result, in this case, mainly depends on this test set. In sum, to be on the safe side, it is better to evaluate your classifier with, e.g., 10-fold cross-validation.

Cheers,

Dr. Samer Sarsam

Thank you for your prompt reply.

What do you mean by "data well-prepared before applying any algorithm" please?

What I meant: normally the training evaluation should give a better accuracy than the test as for the training the output (class) is known. In my case the test accuracy was higher than the training accuracy.

Yes I agree that cross-validation is better to use.

Thanks.

No problem.

Please find my answers below.

What do you mean by "data well-prepared before applying any algorithm" please?

I mean that you need to apply pre-processing techniques before running the machine learning algorithm.

What I meant: normally the training evaluation should give better accuracy than the test as for the training the output (class) is known. In my case, the test accuracy was higher than the training accuracy.

This depends on the utilized learner and its sensitivity to the provided data. Hence, prepare the data nicely before passing it to the algorithm.

Cheers,

Dr. Samer Sarsam

Hi,

I am using seperate supplied training and test data set on Weka.

The situations for the features in the test data are sometimes different to what I had collected with the training data.

What results could I expect?

In my opinion the training accuracy would be excellent but for the test is should be bad.

But Weka results shows for example (with J48):

For training set results:

Correctly Classified Instances 3561 100 %

Incorrectly Classified Instances 0 0 %

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 sitting

1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 walking

Weighted Avg. 1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000

For supplied test results:

Correctly Classified Instances 900 100 %

Incorrectly Classified Instances 0 0 %

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

1.000 ? 1.000 1.000 1.000 ? ? 1.000 sitting

? 0.000 ? ? ? ? ? ? walking

Weighted Avg. 1.000 ? 1.000 1.000 1.000 ? ? 1.000

The test accuracy is perfext here but it should not be.

I wanted to show that when we test with similar siutation the test results should be good, but when you test with different situations the test resulst should be bad.

Any ideas why?

Thank you.

i have question also i attached the test data set and train data set when i applied j48 algorithm it give me error i already attached the screen shot of error would you please help me of this please

Hi,

The problem is in your test data.

The test data is not compatible with the training data.

Please check the attribute: attribute Victims_of_Rape_Total {'LOW ',MEDIUM,HIGH}. In the training is correct but in the test the values are "?".

If you fix this attribute values, it should work.

Best wishes