Hi,

The minimum description length of an object is defined to be minimum number of bits required to uniquely specify that object out of the universe of all objects.

WEKA has two versions of the "Discretize" filter (i.e., supervised and unsupervised). The idea is to discretize a range of numeric attributes in the dataset into nominal attributes.

In order to help you further, here are some questions:

- What is the error message you had?

- What is the problem you are having?

- What exactly you need to do?

- What type of data (instances and attributes) you have?

- How did you work with the Discretize filter?

- Which version of the Discretize filter you used?

- What version of WEKA are you using?

HTH.

Dr. Samer Sarsam

Samer Sarsam

Thanks for sparing time for it.

I have a heart disease dataset, containing 768 instances and 13 , real attributes ( age , sex , chest , resting_blood_pressure ,

serum_cholestoral , fasting_blood_sugar ,

resting_electrocardiographic_results , maximum_heart_rate_achieved ,

exercise_induced_angina , oldpeak , slope ,

number_of_major_vessels , thal , class { absent, present} )

Since, I want to apply the random tree classifier on it, I am trying to discretize the dataset using WEKA Supervised discretization filter (based on Usama M. Fayyad, Keki B. Irani: Multi-interval discretization of continuousvalued attributes for classification learning.)

This discretization filter of weka, returns me the discretized version of the 786 instances by making multiple bins for all attributes except for 4 attributes (resting_blood_pressure ,

serum_cholestoral , fasting_blood_sugar ,

resting_electrocardiographic_results) where it makes just one bin.

The problem is, if some feature in data set contains just one value and we apply some feature selection algorithm like information gain or chi square, this feature will be ranked 0, for it contain just one distinct value.

I want to know, why the discretization method converts all values for the 4 features into just one bin? There is a paper behind the method, but is not easy to understand.

.

when this method produces just one bin for an explanatory variable, it simply means that it has not detected any information in your variable related to your target

you can therefore just remove such variables from your feature selection step

.

by the way, it is clearly explained in your reference ...

.

Muhammad Umer Qureshi,

Please find the answer below.

Q: I want to know, why the discretization method converts all values for the 4 features into just one bin?

Answer

This is because you are using the supervised version of the "Discretize" filter. Means that this filter transforms the numerical variables into a nominal one by using the Class information (i.e., depends on the target class you intend to predict) when selecting the discretization cut points. This approach is called the "Entropy-Based Discretization". The Entropy is calculated based on the class label. It finds the best split in which the bins are pure as possible (majority of the values in a bin correspond to have the same class label). Formally, it is characterized by finding the split with the maximal information gain.

When using WEKA, my advice is to load the whole training set data without removing any feature. Because the process of attribute selection and classification has to be accomplished under the evaluation of 10-fold cross-validation, for instance. To do so, load your data from the "Preprocess" tab. Then, switch to the "Classify" tab. There, you need to select the "FilteredClassifier". Now, configure FilteredClassifier by selecting its "classifier" and "filter" parameters. For the classifier parameter, select AttributeSelectedClassifier. And for the filter parameter choose the supervised version of the Discretize filter (applied on attributes). Now, configure AttributeSelectedClassifier by selecting its classifier parameter (RandomTree, in your case), evaluator: ChiSquaredAttributeEval, and Ranker method as a search method. Click OK to close all the opened boxes and hit the "Start" button.

Here is associated settings that I explained up:

weka.classifiers.meta.FilteredClassifier -F "weka.filters.supervised.attribute.Discretize -R first-last -precision 6" -W weka.classifiers.meta.AttributeSelectedClassifier -- -E "weka.attributeSelection.ChiSquaredAttributeEval " -S "weka.attributeSelection.Ranker -T -1.7976931348623157E308 -N -1" -W weka.classifiers.trees.RandomTree -- -K 0 -M 1.0 -V 0.001 -S 1

NB: Make sure that 10-fold cross-validation is selected. The selected/ranked features can be found in the "Classifier output" panel (existing in the Classify tab) under the "Selected attributes" subsection.

Dr. Samer Sarsam

Fabrice Clerot

Thank you for the answer.

If we believe that the features discretized into just one category have no importance, it sometimes doesn't make sense medically. For example in this case, attributes like

resting_blood_pressure ,

serum_cholestoral,

fasting_blood_sugar ,

resting_electrocardiographic_results

certainly have an important medical importance toward identifying the heart disease. These attributes previously had 47, 144, 2 and 3 distinct values previously.

.

whether it makes sense or not is a different question ; assuming the training setting is correct, one reason why some (a priori useful) attributes are not considered informative may be the small size of the training set

.