Dear Franco, I am a statistician with over 30 years experience and had not heard of the Dunn test. I looked it up and still don't follow it. I then went to my old friend Higgins, Introduction to Modern Nonparametric Statistics, Duxbury and found that my other old friend Bonferroni (see p.93) is used here as well. Suppose we are going to do K comparisons at an overall alpha level of 0.05. Then simply do each of the comparisons at an alpha* level = alpha/[K(K-1)/2]. Since you used a Kruskal-Wallis test, I would probably use something like a Wilcoxon rank-sum test to do the pairwise comparisons each at level alpha* so that the overall alpha level for the package is 0.05. If you want more choices, all of this is in Higgins, sect. 3.3. I have not checked but I suspect that this is easy to run in R. N.B. I believe that you can rundown the p.adjust.method through R help starting with the function that you originally used. If not you might try R online help. You can find this through Google. Best wishes.

Franco and David

Dunn test is implemented in dunn.test package in R as you already knew.

On selecting the right adjustment of p values, Bonferroni is known to be too conservative. There is a distinction in the help document of dunn.test that there are two main types: those controlling FWER or FDR.

Choosing a FDR controller might be better. Here is an article about

Shaffer J.P. (1995) Multiple hypothesis testing, Annual Review of Psychology 46:561-584

Here are the relevant articles from Dunn that this package is reportedly based

Dunn, O. J. 1961. “Multiple comparisons among means.” Journal of the American Statistical Association. 56: 52–64.

Dunn, O. J. 1964. “Multiple comparisons using rank sums”. Technometrics. 6: 241–252.

Mehmet, Thank you. david

Mehmet and David,

thank you both for the useful advices. I will try to set the adjustment of p values as you suggested.

Regards

@David Eugene Booth,

Ack!

The Dunn test (1964) makes sense as a post-hoc test following Kruskal-Wallis. It is similar to a Wilcoxon rank sum test, but it keeps the global rankings from the complete data set used in the K-W test.

You don't want to use pairwise rank-sum tests because they don't retain the information from the complete data set. This is somewhat akin to conducting pairwise t-tests without using the pooled variance. Not a great practice.

For a specific example of the problems you can run in to using pairwise tests on ranked data, look up Schwenk dice. (I need to write up something about this). This is an example of how if using pairwise rank sum tests, you might not be able to order the groups as to which is stochastically greater than the others!