Generally, in a classification task, when the class distribution is unbalance, most classifiers are bias towards the majority class.
Similarly, for Regression, regressors (especially linear regression) usually work better when the target variable is close to a normal distribution (i.e. a bell curve).
My question is: Do the above two general statements stand when a decision tree or random forest based classifier/regressor is used? Is the distribution able to affect/bias the calculation of information gain?
This dilemma of unblanced class can be deal to delete data from background class or foreground class or use SMOTE algorithm.
Hi, Shin,
Not all "decision tree" (DT) methods are the same.
The maximum-accuracy machine-learning DT algorithm employs operations-research (mathematical programming) methods and requires NO distributional assumptions--for raw data or for model residuals. Here are four introductory articles:
Linden A, Yarnold PR (2016). Using data mining techniques to characterize participation in observational studies. Journal of Evaluation in Clinical Practice, 22, 839-847.
https://odajournal.com/2016/09/19/novometric-analysis-with-ordered-class-variables-the-optimal-alternative-to-linear-regression-analysis
https://odajournal.com/2017/04/18/what-is-optimal-data-analysis
https://odajournal.com/2018/11/26/visualizing-application-and-summarizing-accuracy-of-oda-models
ALSO, maximum-accuracy methods may be used to explicitly maximize the accuracy of legacy statistical methods:
https://odajournal.com/2018/10/01/comparative-accuracy-of-a-diagnostic-index-modeled-using-optimized-regression-vs-novometrics
https://odajournal.com/2013/09/20/maximizing-the-accuracy-of-multiple-regression-models-using-unioda-regression-away-from-the-mean
- Badges
- Science topic