Hi,

I think that the main advantage of such methods is the simplicity of the implementation. Another advantage is the readability, a tree is easily understandable by human beings. The kernel methods are like black box, even if they are efficient in certain cases. They are complex and need a certain knowledge background in order to understand them. For a simple problem (linear), tree-based methods are very efficient.

But tree methods also have some issues. When the problem is nonlinear and complex, they are not very efficient. When the number of classes and attributes are high, the tree will be very complex and the construction process can be tough.

hope it will help you.

Regards.

Kernel models assume underlying distribution (Gaussian/gamma etc.) and have to have parameters (mean, standard deviation etc.) which you need to learn for model building. Where as the tree based classifiers can be decision tress, which enforce rules not the parameters. The kernel methods, on the other hand, can be implemented as trees but I believe that was not your question.  

On one hand, kernel method are usually more efficient (in term of predictive accuracy) but on the other hand, trees can be interpretable and that is a huge advantage, especially when working with experts.

AS mentioned by others, one of the primary advantages of the tree models is their human-interpretability. But it has other benefits as well: They can handle non-numeric data, mixed data, categorical data, etc., they can naturally handle missing data; they have lower computational complexity, they can deal with irrelevant inputs, they can perfectly "learn" any training data, etc. In fact, decision trees are perhaps better models than just about any classifier in just about any metric... except accuracy and predictive power. Now you may say, "...well that is one of the most important metrics we care about, what good is a classifier if it does not perform well." And this is where, ensemble systems come in. By using an ensemble of trees, e.g., a random forest or AdaBoost with trees as the base classifier, you get to keep all the benefits and turn the weakness of the tress into a strength.

Kernel based methods have strong predictive power and relatively less likelihood of overfitting, but they do not have the other benefits listed above.

So, at the end, for most supervised classification problems, an ensemble based tree approach is usually a good first choice.

Hello Robi,

I have to temper your answer even if I am a big fan of tree models.

I agree for the computational simplicity, maybe for the non-numerical data handling (whereas I am almost sure that there are kernel based methods that deal with such data), but I totally disagree on one important thing: with ensemble methods you get better predictive accuarcies ok but you loose the interpretation advantage... Maybe I am not aware of some works, but I would really like you to explain me how to interprete a forest?

When talking about trees, one has to mention their lack of robustness: in deed with little changes in your learning data you might get completely different trees. Many researchers avoids trees only because of that drawback. To me, it is not a real drawback because one phenomenon could have many very different explanations.

To conclude dear Supratip Ghose, if you are looking for good predictive accuracies choose kernel based method or ensemble tree methods (check the attributes type as pointed by Robi) but if you are more interested in explanation than in prediction, to interact with experts for example, you definitively should use trees.

Thank you all, specially, Mr Robi Polikar for pointing out  lower computational complexity. Training is scaled as nNlogN in which n is the number of variables and N is the training examples, which makes it relatively fast pointed by others. Subsequent predictions seem also very fast follows this logK pattern for K partitions....

Dr. Nicolas sutton-charani pointed out that it leads to predictive accuracy, however, generic Kernels tends out addressing all predictor variable whereas trees estimate the optimal variable and predictors not suitable should not be chosen during the split.

May you throw you light on the term "interpretability"

Thanks for throwing your perspectives on this question...

Thanks for your participation, especially,  Dr. Robi Polikar, whose article 'Ensemble-based Systems in Decision Making" is a pleasure reading and learning and his participation here is a special treat for me. Mr. Yacine Yaddaden, Dr. Mujahid Sultan, Dr.  Nicolas Sutton-Charani for lending their expertise. 

I agree with you Dr. Nicolas Sutton-Charani with your points on you choice between Tree and Kernel methods, however, I think much of it depends on one's prior knowledge concerning the problem at hand. If this knowledge is well enough for the construction of an  effective Kernel function, then go for kernels...Boosting needs an effective base learners; then it may produce a more suitable result than Kernels...and the interpretability is subjective concerning the problem at hand...However, as pointed by you, instability of the tree for predictions with a few observations and Fragmentation  with the split ...