The main different somehow is about the domain of application. Note that kNN and SVM are used for continuous value inputs, unlike Decision Trees that is applicable for continuous and categorical inputs. If you deal with a problem where inputs are categorical values ( ~ discrete values ) even in part then you have to apply the trees.

The main advantage is interpretability. Decision trees are "white boxes" in the sense that the acquired knowledge can be expressed in a readable form, while KNN,SVM,NN are generally black boxes, i.e. you cannot read the acquired knowledge in a comprehensible way. You can read Michalski on the topic.

thanks for your reply. I want exert Decision tree for my project.

The main different somehow is about the domain of application. Note that kNN and SVM are used for continuous value inputs, unlike Decision Trees that is applicable for continuous and categorical inputs. If you deal with a problem where inputs are categorical values ( ~ discrete values ) even in part then you have to apply the trees.

Two quick notes:

1) In terms of decision trees, the comprehensibility will depend on the tree type. CART, C5.0, C4.5 and so forth can lead to nice rules. LTREE, Logistic Model Trees, Naive Bayes Trees generally are less so. They are running models within each node. In this case, the latter are using a divide and conquer approach, merged with 'modeling'. CART and the C5.0/C4.5 family are using unit tests, which lead to the comprehensibility.

2) KNN, at least, can be used with categorical data and/or mixture of continuation and/or categorical. This can impact the distance measure utilized. You could look at D. Randall Wilson's Advances in Instance-Based Learning Algorithms (dissertation) or Improved Heterogeneous Distance Functions (journal) for examples. Granted, a bit old, but the point is still valid. Papers can be seen here: http://synapse.cs.byu.edu/~randy/misc/pubs.html

I'd like to add that a tree does automatic feature selection, is faster to build and has less parameters to tune.

Which one gives the best model depends on the data. No free lunch.

http://expdb.cs.kuleuven.be/expdb/query.php here you can find a large collection of experimental results. SVM with a polynomial kernel and C4.5 are among the top performing algorithms.

for the type of input data, i can say that CART algorithm can handle continuous data and of course can be used in regression problems.

These are pros of DT:

+ Ability of selecting the most discriminatory features.

+ Comprehensibility so that can be used in Rule Generation problem

+ Data classification without much calculations

+ Dealing with noisy or incomplete data

+ Handling both continuous and discrete data (you have to choose proper algorithm)

Cons:

- The high classification error rate while training set is small in comparison with the number of classes

- Exponential calculation growth while problem is getting bigger.

- Need to discrete data for some particular construction algorithm.

In an analysis, I found NN is best in terms of predicting power compared to logistic regression and DT. In the DT Analysis all the continuous predictors were not psrt of the DT, only the categorical variables remained. Does this count as a shortcoming of DT? 

Shortly, the advantages of decision trees (DT) are the following:

– Implicitly perform feature selection.

– Discover nonlinear relationships and interactions.

– Require relatively little effort from users for data preparation:

• they do not need variable scaling;

• they can deal with a reasonable amount of missing values;

• they are not affected by outliers.

– Easy to interpret and explain.

– Can generate rules helping experts to formalize their knowledge.

You can also use ensemble of decision trees, e.g.,:

Random Forest

C5 combined with boosting

stochastic gradient boosting, e.g. xgboost )the winner of many Kagle competitions)

Thank you for the answer. Is there any specific case where NN will be more applicable? As I have found out, NNs being black boxes are less useful for understanding meanings, but due to the scope of getting better acclimatized to the data through training iterations, it can have more predictive power. So which one is better?