We're planning to conduct a data mining study which will be using ID3, C4.5 and CART algorithms. Naturally, this would then generate three different decision trees.
Would it be possible to create a random forest that uses these three different models?
We're planning to use Python's orange and scikit-learn for this.
Short answer: yes, it would be possible.
Longer answer: you'd be doing extra work, quite possibly for no added benefit. The extra work would be to randomize the tree algorithm for each tree in the forest, as well as bootstrapping the data and selecting the subset attributes for each node in each tree. This would be worth the effort only if you thought that the resulting predictive model would be better than those derived using ID3, C4.5 or CART individually. These algorithms differ in their approaches to pruning more than anything else, and pruning isn't done in random forests, so I would expect model A (using all three) to be very similar to model B (using just ID3, say).
It is an easy experiment to run, though, if you have the time.
Short answer: yes, it would be possible.
Longer answer: you'd be doing extra work, quite possibly for no added benefit. The extra work would be to randomize the tree algorithm for each tree in the forest, as well as bootstrapping the data and selecting the subset attributes for each node in each tree. This would be worth the effort only if you thought that the resulting predictive model would be better than those derived using ID3, C4.5 or CART individually. These algorithms differ in their approaches to pruning more than anything else, and pruning isn't done in random forests, so I would expect model A (using all three) to be very similar to model B (using just ID3, say).
It is an easy experiment to run, though, if you have the time.
You can also design a multi-scheme classifier that uses any number of algorithms as committee to output one classification result based on their individual outputs.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python