In my opinion, well-known classification methods (NN, SVM, LogRegr., etc.) work good only for tasks with continuous variables or discrete variables with a lot of possible values (e.g., “age”). If most of the variables are discrete with small amount of possible values (e.g., 2…4 values), building of the continuous separate line by means of some kernel will get large error, isn’t it? So, question is following – what classification methods should we use for tasks, in which most (or even all) of the parameters have a small amount of possible values ?
Thanks beforehand for your answer. Regards, Sergey.
Hi Sergey,
decision trees and ensemble of decision trees should be able to handle such variables (including categorical variables). Popular ensemble algorithms, which are more powerful than single trees, include bagging and different boosting algorithms.
Greetings, Sebastian
Dear Sebastian, thanks a lot for your fast answer. These algorithms are applicable for tasks both with continuous and discrete variables, or only for tasks with discrete variables ? Regards, Sergey.
They are applicable with continuous and discrete variables.
Greetings, Sebastian
But how to select optimal values of boundaries (logical conditions : if < X, if > X,...) on these trees ? Foe example, tree with 10 gates (logical switches), eaxh switch has 10 possible values. It will be 10^10 possible combinations for values of control parameters. How to select them to optimize some measure ?
Alex, thanks for good question. But in my opinion, it is possible to build less expensive tree. For 8 possible values of each feature it is necessary only 3^10 combinations - 3 binary switches for each feature. Regards, Sergey.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification