In research, I have a set of random forests which are generated from a given set of datasets. Now I would like to calculate the description size (based on the MDL method proposed by Quinlan) of each of these random forests. Do you have any idea how I can calculate it?
Dear Paul,
I want to calculate the description length of a certain random forest. First thing that come to mind is that we can calculate description length of each trees and then add them up, but it is not helpful because the strength of a random forest is the power of majority voting, which is not included in this calculation. Besides, the correlation between trees (which has a effect on the random forest strength) is not included in this calculation. So, I need a method to include all these characteristic measures of a random forest and return a unique real number ( I would like to use this number as the measure of size complexity of the random forest)
It seems like you're asking for something from MDL that MDL isn't intended to give. MDL is an encoding/compression scheme that after decoding/decompression would give you the original forest back, with the same behaviour it had before it was encoded. Additionally encoding the behaviour or characteristics of the behaviour of the forest seems redundant.
Paul Mayer, yes indeed, MDL is a great measure for measuring the complexity of a model, and I'm sorry if my answer gave you a different impression. My observation is merely that the behaviour of the encoded model is inherently encoded with the model, and an external representation of behaviours (like the voting) to be encoded into the MDL would be redundant.
I agree that some other approach either as a complement or substitute for MDL would be appropriate for describing everything that Pouya Sinaian wishes to reify.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Learning