If we see the random tree built in method of the weka tool, it says, It is a "Class for constructing a tree that considers K randomly chosen attributes at each node".
I want to know if it randomly select attributes at each level of the tree, that why is that, it always generate the same tree and yields exactly same classification results for a particular set of attributes?
Hi,
It indeed selects a K number of attibutes to compute the entropy at each node. The fact that you have exactly same tree is due to the seed for random numbers (S parameter), which by default is 1. Should you change that parameter and you will have different results.
Regards!
Ricardo Almeida
Do you want to say that behind the scene, it is using the entropy (Information Gain) as the metric for splitting at each tree level?
I'm not sure if Weka uses Information Gain or Gini Gain to split the nodes. It's accepted that neither metric is better than the other. Gini computation tends to be faster, though.
Regardless of the metric, let's assume that you have 10 attributes and you set your K as 5. At each node, instead of computing the Gain metric of each one of 10 attributes to choose the best one to split the node, the algorithm chooses 5 attributes randomly and then compute the Gain metric only for the 5 chosen to perform the split.
This approach is particularly interesting if you build several Trees (Random Forests) since you increase diversity among the models and (in general) improve generalization performance.
Regards.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification