Hello,
I'm trying to model my dataset with decision trees in Python. I have 15 categorical and 8 numerical attributes. Since I can't introduce the strings to the classifier, I applied one-hot encoding to the categories however some of the categories have 40 or above levels. Therefore dimensionality increased too much. Except the computational cost and overfitting possibility, I also realized that the categories are treated like continuous variables despite being one-hot encoded. For example, I saw silver
Dear Gul,
I suggest you follow deeply :: https://towardsdatascience.com/understanding-decision-tree-classification-with-scikit-learn-2ddf272731bd
https://scikit-learn.org/stable/modules/tree.html
https://www.kaggle.com/gabrielaltay/categorical-variables-in-decision-trees
if your categorical data is not ordinal then we can use sklearn.preprocessing.OneHotEncoder or pandas.get_dummies, but it is computationally expensive.
you can try few more options :
Spark - you can use the decision tree classifier in Spark where you can explicitly declare the categorical features and their orders too.
weka - Does not (needs one hot encoding) H2O - http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/gbm-faq/histograms_and_binning.html
Thanks,
Alok
I hope you have built a pipeline for categorical and numerical data and it's corresponding transformations i.e one-hot encoding and scalers.
Plus if you have more than 40 levels, then you might as well use it as a numerical value instead of keeping it categorical. Infact this approach might work for caterories with values as many as 7-10. There are publications that have treated likerts scale(categorical with 5-10 values) with parametric tests with success as would a non-parametric test.
- Badges
- Science topic