I am working on a classification problem. I have a dataset containing equal numbers of categorical variables and continuous variables. How do I decide which technique to use, between a decision tree and logistic regression?
Is it right to assume that logistic regression will be more suitable for continuous variables and that decision trees will be more suitable for both continuous and categorical variables?
You are right. Generally, the presumptions of the machine learning methods are much less than statistical methods.
Hello Hewa,
I'd suggest that logistic models are fully capable of incorporating categorical IVs, just as classification/decision trees are.
About the only potential advantage of classification/regression trees is that the software for these methods usually considers the possibility of interactions among the relevant IVs you have, whereas a regression model requires that the analyst explicitly include the interaction terms as IVs in order for them to be considered.
As classification/decision trees entertain many possible models, these can be highly opportunistic and possibly overfit the data in hand. For this reason, model validation (usually on separate data) is strongly recommended. It's not a bad idea for any type of model-building exercise!
Good luck with your work..
Hi,
Decision Trees are non-linear classifiers; they do not require data to be linearly separable. When you are sure that your data set divides into two separable parts, then use a Logistic Regression. If you're not sure, then go with a Decision Tree.
Moreover, Decision trees are more likely to overfit the data since they can split on many different combination of features whereas in logistic regression we associate only one parameter with each feature.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence