I want to calculate the effect sizes of Wilcoxon and Mann-Whitney tests. Can I use the formula r=Z/sqrt(N) when the samples are smaller than 20 (where N is all the observations) ?
To add to Jochen's reply, effect size is indeed defined for the Wilcoxon Mann-Whitney test. The underlying test statistic, U, can easily be rescaled to give the probability that an observation from one group will be higher than an observation from the other. This measure of effect size is variously known as the common language effect size, the area under the ROC curve, Harrel's c (though C is a special case) etc.
But it is a very useful measure of effect size indeed - in a clinical trial, for example, it corresponds to the probability that a person on the new treatment will do better than a person on the comparison treatment.
To calculate it, simply divide U by its maximum value, which is the product of the Ns for the two groups
I've a little paper on this, which is still alas behind a paywall, but will pretty soon go free. http://www.stata-journal.com/article.html?article=st0253
"effect size" is not defined when you abandon the data and use the ranks. If you need this for detemining the sample size required to achieve a power for the test (of a location shift), then you can use bootstap methods (there cannnot be a general solution since there is no general distributional model that could be used!). Thus you have to have pilot data for the distribution under H0.
Possibly read:
Biometrics. 1988 Sep;44(3):847-60.
Estimating the power of the two-sample Wilcoxon test for location shift.
To add to Jochen's reply, effect size is indeed defined for the Wilcoxon Mann-Whitney test. The underlying test statistic, U, can easily be rescaled to give the probability that an observation from one group will be higher than an observation from the other. This measure of effect size is variously known as the common language effect size, the area under the ROC curve, Harrel's c (though C is a special case) etc.
But it is a very useful measure of effect size indeed - in a clinical trial, for example, it corresponds to the probability that a person on the new treatment will do better than a person on the comparison treatment.
To calculate it, simply divide U by its maximum value, which is the product of the Ns for the two groups
I've a little paper on this, which is still alas behind a paywall, but will pretty soon go free. http://www.stata-journal.com/article.html?article=st0253
Although I've not read them yet, I see that Robert Newcombe's book on confidence intervals for proportions & related measures (http://www.crcpress.com/product/isbn/9781439812785) has a chapter called "Generalised Mann-Whitney Measure" and another called "Generalised Wilcoxon Measure" (chapters 15 & 16 respectively). The original poster may find chapter 15 useful.
Thank you! It is an entirely different kind of "effect". Mathematically this is perfectly fine (I think), I am only a little itchy about the fact that probabilities itself are used as effect-size measure (this is related to the meaning of probability, what is not so clear, though).
I'm comparing between two independent groups (control and Treatment), the treatment has higher mean ranks compared to control group. Can I say a significant effect of Group (The mean ranks of control and treatment group were 5.2 and 9.3, respectively)", p < 0.05, r= -.57. There is a medium effect observed of the intervention on treatment group.
I can just add one thing all didn't mention. The most robust measure of effect size I found is the nonparametric common language effect size (A) which can be interpreted in a simple language by telling the probability that a random value taken from group 1 will be higher than group 2, so as A come closer to 0.5 this means that there is no difference even if ranksum p-value is < 0.05. I have a function to find A in matlab and it's easy to be computed in any other language.
According to Fritz, Morris, and Richler (2011) we can calculate effect size for the Mann-Whitney U-test using the the mentioned formula. The following website is realy helpful:
Thank you for this thread. Do you think this formula could be used for within subject repeated measures? Or should a factor be added to consider the extent of estimated progress. Without an a priori estimation of the expected treatment effect, the instruments’ ability to measure change cannot be disentangled from the treatment effect. I am concerned about the overstatement of treatment results with the use of the Goal Attainment Scale when one would expect some progress in attainment of goals that were the focus of the intervention.