According to the Friedman's Stochastic Gradient Boosting algorithm, the variable space is divided into J regions if the regression tree has J leaf nodes. But Hastie et al. suggested that J > 10 is unlikely to be required (from his empirical experiments). But what I think is, if we divide the variable space into more number of regions then we will get more accuracy in fitting the residuals ( we may also face overfitting..). I know it is an empirical statement given by Hastie et al.. But what is the actual logic behind this concept? Should we always check for higher number of leaf nodes or we should blindly choose a number in between 4 to 8? If we are getting a better prediction model using more than 10 leaf nodes then up to what extent I should check for getting the exact number of leaf nodes for my regression tree that gives the best prediction model for my particular data set?
The number of branchings is roughly equivalent to the number of variable interactions in any one tree, similar to how multiple variable terms function in a linear regression. Much of the richness in the decision boundaries is captured by voting among multiple trees rather than by the depth of the trees.
The number of branchings is roughly equivalent to the number of variable interactions in any one tree, similar to how multiple variable terms function in a linear regression. Much of the richness in the decision boundaries is captured by voting among multiple trees rather than by the depth of the trees.
take a look in this page
http://fr.openclassrooms.com/informatique/c-1/cours
Dr. Agosta,
As far as I know, voting is used only in Random Forest Algorithm for classification, therefore I am unable to relate your answer to my question.
Dr. Albrecht,
I agree with your statements but I don't see how this answers my question. Could you please clarify?
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining