If you use 10-fold cross validation to derive the error in, say, a C4.5 algorithm, then you are essentially building 10 separate trees on 90% of the data to test on 10% - 10 times. How can you create a single representation from all those ten trees created?
For example - how does WEKA and Rapidminer give me a single tree after cross validation on a C4.5?
I created all the trees in R, but I don`t know how to visualize the result as a single tree.
.
i do not know the specifics of R or Weka but the "tricky" part here is that the CV procedure is used as a proxy for the error rate of the tree built from the full training set.
this is why only one tree is given as a result : this tree is grown from all the training data ; it's error rate is estimated from the CV procedure which indeed grows 10 trees from 9 splits of the training set out of 10 (so from 90% of the training set "only") and estimates their error rates from the remaining split (10% held out data).
thinking about it, i wonder which model would have the best performance (on held out data, of course) : the (very) small random forest of the ten trees built during the CV procedure or the tree built from the full training set ? Of course, the forest is not a tree so if you want a tree at the end, the question is just for speculation ...
.
Thank you both for the clarification, that helped me a lot.
So as I got, this is the advantage of k-fold cross validation to Split validation:
When using k-fold cross validation, we can train the tree using all the data we have, so there is no loss of information, and at the same time we can evaluate the model because of the k-fold procedure.
But using the split validation, we have to divide our data into train and test set, and build our tree just on the training set, right?
Can you also tell me how WEKA train the tree when using Split Validation?
.
as i said above, i don't know how weka does it
when i use split validation (that is, i have two disjoint sets of data, training set and validation set), "the" tree (that is, the tree i shall report as my model in the end) is trained on the training set and its generalisation performance (error "out of training set") is estimated from the error on the validation set. I do not retrain a tree on the full dataset since i have already a tree (trained on the training set) with a nice estimate of its generalisation error. Of course, this wastes some data.
note that in the SV case, the error is indeed the error of "the" tree on the validation set whereas with CV, the error of "the" tree is estimated from the errors of the 10 trees (other, different trees !) built on 9/10 splits and evaluated on the remaining 1/10 split.
.
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining