You need to preprocesses your train and test datasets.  first choose the train file, and apply "choose/unsupervised/attribute/string to word vector" and save the dataset with a different name.  Repeat the same process for the validation dataset.  Then, use the new datasets for your experiments.

Hi Uvernes, 

I do appreciate your suggestion, but it will not work accurately due to the possibility of having different attributes with different orders (and this what happened in the posted question). Thus, the only one solutions is filtering training and test sets internally. I did that and this is the results I had:

orrectly Classified Instances 85                68.5484 %

Incorrectly Classified Instances 39           31.4516 %

Kappa statistic 0.3234

Mean absolute error 0.3969

Root mean squared error 0.4472

Relative absolute error 80.1549 %

Root relative squared error 89.866 %

Total Number of Instances 124

=== Detailed Accuracy By Class ===

TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class

1.000 0.696 0.636 1.000 0.777 0.439 0.652 0.636 watery

0.304 0.000 1.000 0.304 0.466 0.439 0.652 0.618 blue

Weighted Avg. 0.685 0.382 0.800 0.685 0.637 0.439 0.652 0.628

=== Confusion Matrix ===

a b

I see, then you can combine both datasets (must probably with a text editor, MS Excel, or any similar tool) , Load the combined dataset in WEKA and shuffled it, then using WEKA, extract /save a subsample (let say 2/3)  as your new training set and extract/save another subsample (let say 1/3) as you new validation data set.  Now the two datasets should have same attributes with same order.  Conduct your experiments in the new datasets.

Hope this help.

Vahideh,

The answer is NO. I didn't use the "Percentage split" option because, simply, it does't use the external test set data.

HTH.

Dr. Samer Sarsam

Samer,

So, would you please explain step by step, how did you filtered them internally? Thanks.

(I tested the link below  and used cmd, but there is no change and again there is the same error.)

https://thekandyancode.wordpress.com/tag/training-and-test-set-are-not-compatible/

Vahideh,

Well, I've designed two flows to accomplish this; first one builds the model on the training set, then saves this model. Second flow is used to call this model and apply it on the test set, and finally produce overall summary statistics.

NB: The resulting model belongs to a pruned "J48" classifier after converting the data into a numeric type using "StringToWordVector" filter. 

HTH.

Samer

Hi Vahideh,

Having such error is expected because of the wrong procedure you performed. Precisely, you've applied "StringToWordVector" from the "Preprocess" panel, and it converts the training set from string type into a numeric one.Then you applied the saved model that handles numeric type (i.e., generated from the training set) on the test set (string type) that gets loaded from "Supplied test set" option, which produces the error you received--having different data types (numeric training set and string test set) instead of having only one type.

HTH.

Dr. Samer Sarsam

please i want tutorial video to apply this steps in weka gui easly

Uvernes A Somarriba please indicate your answer about this error

Hi Yasmine Ali

You are getting such error because your training set is different from the test set. Both datasets have to be compatible in terms of features' name, number ,order, and type.

HTH.

Dr. Samer Sarsam

how can i solve it?

Samer Sarsam

Yasmine Ali ,

You need to redesign your data again with considering the similarity in features (of training and test sets) that I have highlighted in my previous post.

Dr. Samer Sarsam

Hi,

As Dr. Sarsam mentioned before, this error happens because I changed "string to word vector" in preprocess. And that is wrong. The correct way is:

step 1: open train file.

step 2: go to classify.

step 3: go to Choose/ meta/ FilteredClassifier. And then click on that part, so a window opens. There you can choose your classifier (naive bayes, for example) and in the "filter" part go to filter/ unsupevised/ attribute/ stringToWordVector.

Step 4: choose "use training set".

Step 5: save the result.

step 6: choose "Supplied test set".

step 7: right click on the result and choose "Re-evaluate model on current test set.

Vahideh Torabi please show me figure indicate sample for train file and test file or video

does train set tweet only not vectors?

does test set tweet only not vector?

Yasmine,

I've enclosed the sample files in the question before. Go upper you can find them.

The files are text. They contain sentences and because of that we should choose "stringToWordVector" so that weka changes the sentences to binary vectors. So the steps that I wrote before, are just related to text files and if your files are vector, it's not necessary to do these steps.

How can I make a training and testing file the same number of features?

Yasmine Ali

Well, let's take the example data of Vahideh.

Both training and test sets are already containing the same number of features (Text and class). Basically, those features need to be declared at the very early stage in the header of the ARFF file.

The following is an example feature declaration in Vahideh 's ARFF files header:

@attribute Text string

@attribute class {watery,blue}

If your training and test data have different headers (regardless to the relation name: @relation 'abi train' or @relation 'abi test') that might cause the error you had. In this case, you need to redesign data again and consider providing same header for both datasets. Another reason behind your error is that you may have applied the StringToWordVector filter in the "Preprocess" panel and then used FilteredClassifier in the "Classify" panel.

Finally, without seeing your data, detecting the error only from a picture is not really useful. Hence, if you still have any issues, just attach your data to help you further.

Cheers,

Dr. Samer Sarsam

Samer Sarsam

Uvernes A Somarriba

Vahideh Torabi

the error because convert train and test file to vector?

You can make bag of word model using your test file, then use that bag of word model vectors in weka.

you can use n fold cross validation in weka, it will decide training and testing at its own.

The accuracy and result of cross validation in weka one file for training and testing will high performance more than two separate work files?

this is easy for me

@Munawar

Yasmine Ali

Your datasets are designed correctly, but the model that you attached is wrong. The attached model is a *pure* SMO model, while the the one you used mainly seems to be different (I presume that the main model is based on the combination: FilteredClassifier, SMO, and StringToWordVector). Therefore, I have attached the right model to be loaded and used.

Cheers,

Dr. Samer Sarsam

Yasmine Ali

Find the answer below.

The accuracy and result of cross validation in weka one file for training and testing will high performance more than two separate work files?

Answer

If you provide an independent test set (like what you did), the performance of the algorithm will depend highly on the test set content; hence, it is not recommended to evaluate the performance of an algorithm using only an independent test set.

In general, k-fold cross-validation (e.g., stratified 10-fold cross-validation applied in WEKA) is recommended to be used at the very early stage. This approach gives an overall idea on your model performance since it automatically performs training, testing, and validation. Then, if you like the performance of a model, save this model and use it again on your test set.

HTH.

Dr. Samer Sarsam

thank you dr samer very much

Java.io.IOExeption: 2 problem on line: 196 this error made when I input dataset.csv in weka to convert it into .arff my file text tweet for classifiation?

please help

Yasmine Ali

I have fixed your data and attached it back here, so kindly find the attachment.

Cheers,

Samer Sarsam

how can fixed data or cleaning text tweet before load them in weka pleeeeeease ?

thank you doctor Samer Sarsam

Yasmine Ali

If you meant the data to be loaded at the very early stage into WEKA, then you need make sure that loaded instances are in line with the associated feature type. In other words, all instances in one column belong to the same data type of that column, e.g., numeric. This to avoid having such error you had -- Java.io.IOExeption: 2 problem on line: 196.

Cheers,

Dr. Samer Sarsam

The meaning of cleaning text is delete symbols that cause an error in rows when load train set like (" ' , etc)

what is the too use that clean?

Yasmine Ali

To design a proper CSV or ARFF file format, you need to do that yourself; no tool can do that perfectly for you. However, after uploading the data successfully into WEKA, you need to prepare it (data preprocessing/cleaning) using WEKA's techniques.

Cheers,

Dr. Samer Sarsam

not recognized as an 'CSV data files' file. Reason: 1 problem encountered on line : 6 . ?

Please help

Amira Fatin

Your file has many problems, I fixed them and attached it back here with an ARFF format. Just load it into WEKA and it should work without issues.

After having your data loaded, make sure to prepare it nicely (ie, not yet clean) to get reliable result.

HTH.

Dr. Samer Sarsam

I had the same problem and I ended up creating a python module to work with twitter and weka. There are many things you need to clean up in order to load tweets on this tool.

https://github.com/pacu/phron/