Both are non-parametric test that is used to measure the degree of association between two variables. But how are they philosophically different. It would be nice if someone answers with examples and not by just using mathematical derivations.
I will try to explain the difference with an example. It will not be a perfect example, but I hope it will be good enough for a quick demonstration.
Let's say that you want to measure the association between preference of ice-cream (X) and fast food (Y). You ask five respondents (A,B,C,D,E), and they can answer the following way:
1: I hate it, 2: I don't like it, 3: neither like it nor hate it,4:I like it, 5: I love it
Imagine that they provide the following answers:
X Y
A: 2 3
B: 3 2
C: 4 5
D: 2 2
E: 3 3
If you want to calculate the Kendall rank correlation coefficient, or Kendall's tau, you have to examine the pairs of observations (A-B, A-C and so on). Kendall's tau is based on the number of concordant and discordant pairs. A pair is concordant, if xi > xj and yi > yj; or xi < xj and yi< yj. In the example above, we have five concordant pairs: AC (because 4>2 and 5>3), BC (4>3 and 5>2), CD (4>2 and 5>2), CE(4>3 and 5>3) and DE (3>2 and 3>2). A pair is discordant if xi > xj and yi < yj; or xi < xj and yi > yj. In our example AB is a discordant pair. Pairs that are neither concordant nor discordant are ties.
Kendall's tau has three different variants, Tau, Tau-b and Tau-c. In each variant, the numerator is the difference between the number of concordant and discordant pairs. The variants differ in the denominator, because they handle tied pairs differently.
Spearman's rank correlation coefficient (rho) calculated by applying a Pearson correlation on the ranked data, not the actual data themselves. So to answer your question, Kendall's tau examines the difference between concordant and discordant pairs, while Spearman's correlation is basically a Pearson correlation on ranked data.
Kendall's tau is usually lower than Spearman's rho, and I think I have read somewhere that Kendall's tau is insensitive to error and is more accurate with smaller sample size, but I am not sure on this.
I will try to explain the difference with an example. It will not be a perfect example, but I hope it will be good enough for a quick demonstration.
Let's say that you want to measure the association between preference of ice-cream (X) and fast food (Y). You ask five respondents (A,B,C,D,E), and they can answer the following way:
1: I hate it, 2: I don't like it, 3: neither like it nor hate it,4:I like it, 5: I love it
Imagine that they provide the following answers:
X Y
A: 2 3
B: 3 2
C: 4 5
D: 2 2
E: 3 3
If you want to calculate the Kendall rank correlation coefficient, or Kendall's tau, you have to examine the pairs of observations (A-B, A-C and so on). Kendall's tau is based on the number of concordant and discordant pairs. A pair is concordant, if xi > xj and yi > yj; or xi < xj and yi< yj. In the example above, we have five concordant pairs: AC (because 4>2 and 5>3), BC (4>3 and 5>2), CD (4>2 and 5>2), CE(4>3 and 5>3) and DE (3>2 and 3>2). A pair is discordant if xi > xj and yi < yj; or xi < xj and yi > yj. In our example AB is a discordant pair. Pairs that are neither concordant nor discordant are ties.
Kendall's tau has three different variants, Tau, Tau-b and Tau-c. In each variant, the numerator is the difference between the number of concordant and discordant pairs. The variants differ in the denominator, because they handle tied pairs differently.
Spearman's rank correlation coefficient (rho) calculated by applying a Pearson correlation on the ranked data, not the actual data themselves. So to answer your question, Kendall's tau examines the difference between concordant and discordant pairs, while Spearman's correlation is basically a Pearson correlation on ranked data.
Kendall's tau is usually lower than Spearman's rho, and I think I have read somewhere that Kendall's tau is insensitive to error and is more accurate with smaller sample size, but I am not sure on this.
I am not sure if there is any rule of thumb for this, however I know about two sources that could help. According to Field (2009, Discovering Statistics Using SPSS), Kendall's tau should be used if your dataset is small, and you have a large number of tied pairs. Howell (1997, Statistical Methods for Psychology) suggests that Kendall's tau is overall a better estimation of the correlation. I think Spearman's rho is more popular, because it could be calculated more easily, especially when there were no computers.
While this is not addressing your question, I rarely use any measure of correlation in my data analysis (over the past 30 years), and I sometimes skip this topic in some of my statistics courses. I often use regresion analysis in applications where a correlation coefficient could be calculated too. Good luck with your research.