A Mann-Whitney-Test is often used as an alternative to the t-test when data is not normally distributed. However, both test different hypothesis. While the t-test compares means of the groups, the MWU (Wilcoxon rank-sum test) tests the median of the difference between a sample from both distributions (not for differences in medians, as is often thought).
I have two experimental groups, which differn in the means of their outcome variable, but not in their medians. The outcome variable is not normally distributed. I used the MWU.
I did write "both distributions (not) differ significantly" but what if I want to indicate the direction of the differences?
Can I use the means of both groups and report the MWU test outcomes? Should I use the medians? Does the latter make sense, if the medians are identical, but still the differences are significant according to the MWU?
Does the latter outcome indicate that something is "wrong" with using the MWU?
Thanks in advance
i do not fully understand what you mean by "the MWU tests the median of the difference between a sample from both distributions". a sample? :-) two samples? or the median of all pairwise differences? that would then be the hodges–lehmann estimate of the difference in central tendency, right?
the way i understand the MWU-test, it can be used with more or less strict assumptions. if the distributions are assumed to be continuous and the alternatives are restricted to location shifts then a significant test result can be seen as either a non-zero difference between the two medians or between the two means or even a non-zero difference in central tendencies (as measured by the hodges-lehmann estimate). all of these interpretations are equivalent under the assumptions stated.
under less strict assumptions (in essence independent ordinal data), the MWU-test just checks if two distributions could possibly be identical. if, in this case, you accept h0 then this implies that both means and medians might also be identical, but if you reject h0 then "everything" is possible, even different means and different central tendencies but identical medians.
If data are skewed you can't use mean and SD to describe the shape of distribution. So it doesn't make sense to report mean difference while using MWU. MWU is to test difference of entire distribution based on rank. You can report Median and IQR of each group to show the difference in distribution.
For skewed data, MWU is unlikely to test medians either, especially when the shapes of the distributions are not identical. The test is about stochastic equivalence: the null hypothesis is Pr(a>b) = 0.5 for a and b being random values of group A and B, resp. If you reject that hypothesis, then you say your data is sufficient to "see" that either Pr(a>b) < 0.5 or that Pr(a>b) > 0.5. I'd say that you can check the direction using the empirical estimate of that probability.
Thanks for the answers so far.
A further note:
Are violin-plots, instead of mean bar graphs, a good visual representation in order to visualize differences indicated by MWU?
Depends on the number of values in the groups. I'd say: for few values (
i do not fully understand what you mean by "the MWU tests the median of the difference between a sample from both distributions". a sample? :-) two samples? or the median of all pairwise differences? that would then be the hodges–lehmann estimate of the difference in central tendency, right?
the way i understand the MWU-test, it can be used with more or less strict assumptions. if the distributions are assumed to be continuous and the alternatives are restricted to location shifts then a significant test result can be seen as either a non-zero difference between the two medians or between the two means or even a non-zero difference in central tendencies (as measured by the hodges-lehmann estimate). all of these interpretations are equivalent under the assumptions stated.
under less strict assumptions (in essence independent ordinal data), the MWU-test just checks if two distributions could possibly be identical. if, in this case, you accept h0 then this implies that both means and medians might also be identical, but if you reject h0 then "everything" is possible, even different means and different central tendencies but identical medians.
If you have two groups with not normally distibuted data, I would compare at first the shapes of these two distributions e.g. with the two-sample Kolmogorow-Smirnow test (with Lilliefors correction). In the case, that the shapes of the two distributions can be considered as the same, the U-test of Mann/Whitney/Wilcoxon for comparing the medians should follow. If the shapes of the distributions are different other tests should be used (e.g. a permutation test).
Because the two-sample Welch test is very robust against non-normality, you better use this test. Combining two tests applied to the same data gives enormous complications, it is not possble two calculate the overal risks of both tests. See
Rasch D. and Guiard, V. (2004)
The Robustness of Parametric Statistical Methods
Psychology Science, 46 ; 175-208
268Kubinger, K.D., Rasch, D. & Moder, K. (2009). Bericht: Zur Legende der Voraussetzungen des t-Tests für unabhängige Stichproben. Psychologische Rundschau, 60, 26-27
Rasch, D., Kubinger, K.D and Moder, K (2011).The two-sample t test: pre-testing its assumptions does not pay off Statistical Papers , Volume 52, Number 1, 219 - 231
Rasch, D. und Schott, D. (2016) Mathematische Statistik, Wiley – VCH, Weinheim
I agree with the answer of D. Rasch in principle, that there is a problem if you apply two tests on the same data, because you don't know the overall risk. Of course it would be better to test assumptions for tests on different data, but it is very often not possible in practice. You need as well informations about the distributions as very often assumptions about the population variances. For using the two-sample Welch test, you also should know if the two population variances are not equal. Otherwise you would prefer the normal two sample t-test. But in any case it is useful to have a look to the two distributions.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics