You need to insert the "class" attribute in your test set (in spite of being the target attribute). Insert "class" at column A with the value "?" for all instances of your test set

Dear Concalves

I tried your suggestion but the problem do not solved.

Hi,

I've used a method like NB (you can use SMO) to label the unlabeled instances. Then I placed those prediced labels back to each instance in the test set. It worked perfectly with 100% accuracy. I believe after labeling the instances, your dataset was not well-structured.

HTH.

Samer Sarsam, PhD.

First, I saved both files as ARFF files. Important: you have to specify the real classes at your test set, otherwise, Weka will not be able to compute the accucary, confusion matrix and the other measures associated to the evaluation of the test set.

Then I conducted the experiment the following way:

1-Open the "train" dataset in Weka. Next, choose "class" as the class attribute ("Preprocess" tab)

3-Click at the "Classify" tab. Once more, I had to choose "class" as the class attribute (I selected just below the "More Options" button).

4-Choose the SMO classifier ("Choose" button)

5-Click at option "Supplied Test Set" and select your "test" dataset. IMPORTANT -> Before closing this window you have to inform Weka that "class" is your class attribute in the test set

6-Then, just click at "Start" button and it is going to work.

Dear Dr. Gonçalves

Thank you so much for your informative and helpful answer. The problem is solved .

Best Regards

Dear Dr. Gonçalves

Thank you so much for your informative and helpful answer. The problem is solved .

Best Regards

Ok, I'm glad it works!

Best Regards

This is the result:

===============================

p1

mean 8.1579 10.955

std. dev. 0.6665 0.2355

weight sum 20 20

precision 0.1038 0.1038

p2

mean 7.7354 9.2316

std. dev. 0.3637 0.2322

weight sum 20 20

precision 0.0759 0.0759

p3

mean 7.395 7.9445

std. dev. 0.5144 0.2512

weight sum 20 20

precision 0.0572 0.0572

Time taken to build model: 0 seconds

=== Predictions on test set ===

inst#,actual,predicted,error,prediction

1,2:no,2:no,,1

2,2:no,2:no,,1

3,1:yes,1:yes,,1

4,1:yes,1:yes,,1

5,1:yes,1:yes,,1

6,2:no,2:no,,1

7,2:no,2:no,,1

8,2:no,2:no,,1

9,2:no,2:no,,1

10,2:no,2:no,,1

11,2:no,2:no,,1

12,2:no,2:no,,1

=== Evaluation on test set ===

Time taken to test model on supplied test set: 0.02 seconds

=== Summary ===

Correctly Classified Instances 12 100 %

Incorrectly Classified Instances 0 0 %

Kappa statistic 1

Mean absolute error 0

Root mean squared error 0

Relative absolute error 0 %

Root relative squared error 0 %

Total Number of Instances 12

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 yes

1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 no

Weighted Avg. 1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000

=== Confusion Matrix ===

a b

Dear Dr. Sarsam

Thank you so much it was great help

Dear Mokhtar Nosrati, I have just tried Eduardo Corrêa Gonçalves 's suggestion on your files, but I am still obtaining:

Time taken to build model: 0 seconds

=== Evaluation on test set ===

Time taken to test model on supplied test set: 0 seconds

=== Summary ===

Total Number of Instances 0

Ignored Class Unknown Instances 12

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

? ? ? ? ? ? ? ? yes

? ? ? ? ? ? ? ? no

Weighted Avg. ? ? ? ? ? ? ? ?

=== Confusion Matrix ===

a b

Elisabetta Ronchieri,

You are getting an expected output. You will not get any statistical summary in the result if your test set has originally no labels. This is because you have the predicted labels, but you don't have the original ones to compare with.

I have attached the correct design of the training/test set files saved in an ARFF format. To see the predicted labels for the test set, select the choice "PlainText" as choice of the "Output predictions" parameter inside the "More options ..." button. Attached here also the result of NaiveBayes algorithm on the test set (under the name "result") that can be shown in the Classifier output panel.

NB: WEKA by default considers the last column as a class that has to be predicted.

HTH.

Dr. Samer Sarsam

Dear Sarner, thanks for your reply. I obtained what you wrote.

Save the data file as ARFF and then mention the same target class in both (Train and Test)

Samer Sarsam I am getting the same error....

Time taken to build model: 0.11 seconds

=== Evaluation on test set ===

Time taken to test model on supplied test set: 1.81 seconds

=== Summary ===

Total Number of Instances 0

Ignored Class Unknown Instances 18419

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

? ? ? ? ? ? ? ? D

? ? ? ? ? ? ? ? L

? ? ? ? ? ? ? ? C

Weighted Avg. ? ? ? ? ? ? ? ?

=== Confusion Matrix ===

a b c

Ambuj Saxena,

I have fixed the data and attached back here. Then, I applied NaiveBayes that produced the following result:

=======================================================

=== Evaluation on test set ===

Time taken to test model on supplied test set: 0.03 seconds

=== Summary ===

Total Number of Instances 0

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

? ? ? ? ? ? ? ? D

? ? ? ? ? ? ? ? L

? ? ? ? ? ? ? ? C

Weighted Avg. ? ? ? ? ? ? ? ?

=== Confusion Matrix ===

a b c

Samer Sarsam

Thanks for your quick reply, now I understand what I did wrong.

If I don't want to use weka as perdition tool, I saved the result in *.model then where I can see this file?

Once again thanks for your support

You're welcome, Ambuj.

Models generated by WEKA's algorithms can be saved (through the "Save model" option in the "Classify" tab) in the directory that the user specifies.

Dr. Samer Sarsam

i am not getting results when i test the data for classification. How can fix this problem?

Richa

Actually, it is not a problem, you have an expected output because your test set does not have the labels initially.

Dr. Samer Sarsam

Samer Sarsam Thank you

Hi,

I saved a model for document classification by the following steps:

1.WordtoVector filter

2. Attribute selection (in order to increase accuracy)

3. Naive Bayes Multi with 5-fold crossvallidation

4. Save model

Subsequently i have tried to re-evaluate this model on the test set (to predict the class of the test set). Unfortunately the train model and test set are not compatible. i have tried to perform the same wordtovector filter before and at the samen time (using metafilterd classifier)

Any idea what i'm doing wrong?

Thijs Oord

Thijs,

Your process is not correct and might lead to an optimistic result with the 5-fold cross-validation.

On the other hand, the error you had is expected. This is because your training set has different structure from the test set. Make sure that both training and test set are compatible.

Cheers,

Dr. Samer sarsam

Thanks for your answer. So Info gain attribute selection (ranker) is not possible in combination with the string to word vector filter?

No problem.

InfoGainAttributeEval (with a Ranker search method) can be applied in conjunction with StringToWordVector from the Classify tab. In this case, AttributeSelectedClassifier needs to be used as a base learner (classifier parameter) of FilteredClassifier and "StringToWordVector" is the filter parameter of FilteredClassifier.

Cheers,

Dr. Samer sarsam

Thanks to all for helpful responses

Please can anybody preprocess dataset from black blaze hard drive stats for me to classify in weka, I need it from my thesis please

I have the answer if it is needed. Mokhtar Nosrati

Hi, I have two questions please.

Why when i use a separate test set, in the WEKA results window it shows mic.InputMappedClassifier instead of tree.J48 which it normally shows if you apply cross-validation?

Why the accuracy when applying a separate test is less with about 30 % than when applying cross-validation? should not be almost the same? Thank you.

Ziad Salem

The answer to the first question: This is because your training and test sets are not compatible.

The answer to the second question: Because the result depends on the instances in the independent test set data. It has nothing to do with cross-validation results.

HTH.

Dr. Samer Sarsam

Thank you Dr. Samer. In fact I collected two data following the exact process. Then I used one for training and one for test. So they both have exact features, etc. So I cannot see in compatibility in both data. Also, when there are incompatibility in the data we should see ? in the === Detailed Accuracy By Class ===, but this is not the case in my data. No ? at all.

Best wishes. Dr. Ziad Salem

Well, there is a micro details that have to be reconfigured again in your data. This causes the incompatibility between the two datasets. You need to carefully check the structure of your datasets again by yourself.

WEKA is designed to overcome the incompatibility issue by suggesting the use of the InputMappedClassifier method. This to help the users, who don't know how to fix the structure of the data, to perform their tasks.

Dr. Samer Sarsam

Dr. Samer, the structure is exactly the same for sure, I already checked it. there is no differences at all.

Is it possible please to explain what is the " micro details that have to be reconfigured again in your data. "? and also "how to fix the structure of the data"?

Or could you please provide a reference or website about these two comments?

Do you think if the both data are similar we could get this case? Many thanks. Dr. Ziad Salem.

Dear Dr. Ziad,

Usually, if training and test sets are compatible, WEKA will not suggest the use of InputMappedClassifier method.

Compatibility is WEKA's default assumption towards the separated training and test datasets. The term 'micro' is my personal term that refers to the header/data items of the ARFF file format that WEKA uses by default. For more information on the ARFF format, you may consult this link:

https://www.cs.waikato.ac.nz/ml/weka/arff.html

Alternatively, you don't have to have independent test set; a valid assessment of the predictive model's capability be achieved via 10-fold cross-validation. Its results are pretty much reliable.

Dr. Samer Sarsam

Thanks Dr. Samer, Yes it is not needed to have a sperate test set but I wanted to check the results difference with cross validation.

Have you tried deep learning with Weka please? in case your used it, any idea how can we add it to Weka?

If you are interested in human activity recognition, I look forward for future research cooperation. best wishes

In general, I have no issues in joining academic researches, but I need to look at the work first, evaluate its quality, before accepting it.

You can apply deep learning classifiers/filters via WEKA wrappers for Deeplearning4j by installing the "wekaDeeplearning4j" package.

Dr. Samer Sarsam

This is the same for me, the quality of work is very very important to me. I add to this, the work must fall into my field of expertise.

I faced the same issue. In my case, the problem was due to use CSV files. What I do to fix it is the following:

(1) Open in Weka the CSV the training file (Workbench)

(2) Select Save... Store as ARFF

(3) Repeat the process (1) and (2) for the development datasets

(4) and my problem was gone

In my first attemp, weka told me that the labels were not compatible.

(Version of Weka: 3.8.2)

I hope this helps someone

We can use csv files for both training and test but when choosing supply test set you click on more options, then in the classifier evaluation options window you have to choose "plain text" for output predictions instead of the default "NULL".

Now it should work without any problem.

I hope this helps.

José Antonio García-Díaz

The issue is not related to the file format (CSV or ARFF); it's related to the data structure of your training/test set files.

HTH.

Dr. Samer Sarsam

Samer Sarsam

Of course, but I in my case, exporting the file as ARFF solved my issue. And I had two CSVs files with exactly the same structure and data.

Kind regards

José Antonio García-Díaz

You may need to provide your files to check them further.

Cheers,

Dr. Samer Sarsam

Hello,

I tried to perform a data classification in WEKA through the J48 algorithm. There was no problem with the train set and cross-validation, but when I tried "supplied test set" this error appeared and the result was showed as a question mark. The results page is attached. How can I fix this problem?

Diana Iagar

This result is expected, i.e., there is no error, is just that you don't have "labels" for your instances; thus, no testing is accomplished. In other words, you are asking J48 to predict data labels. However, to see these labels, in the “Classify" tab, click the “More options...” button, then choose “PlainText” as an option of the “Output prediction” and apply J48 again (hit the "Start" button).

HTH.

Dr. Samer Sarsam

I did as you said but it is still giving me the same result as you can see in the screenshot.

I attached the datasets I used.

Diana Iagar

No, you don't have the same result, I have highlighed the predicted lables with a red rectangle on your file and re-attached here (see the attached file), under the area:

=== Predictions on test data ===

Cheers,

Dr. Samer Sarsam

You're right. It worked and I managed to do the prediction in Java. Thank you!

Great! All the best.

Cheers,

Dr. Samer Sarsam