Hello,
I have values for two variables with a non-normal distribution. So, the non-parametric Mann-Whitney U test was applied to check for statistically significant differences between the two variables. However, my advisor asked me to use an estimation statistical approach instead of significance testing. Can you suggest me a way to discuss the difference of the two variable distributions?
Thanks!
Did your advisor tell you what kind of analysis would be a "estimation statistical approach"? I'm not familiar with the term.
Thanks for your replies. In fact, the advisor expects to see the effect size of the difference between the two distributions, instead of the statistical significance of the difference between the two distributions. Will the Kolmogorov-Smirnov test work here?
Hello Manjula. If I follow, you have one dependent variable (DV) and two independent groups. And the distribution of the DV is non-normal in both groups.
- What is the DV?
- How are the two distributions non-normal? (Remember that nothing in nature is truly normally distributed.)
- Are the distributions for the two groups fairly similar (e.g., if skewed, are both skewed in the same direction and to about the same extent)?
- What are the two sample sizes? (As Ette Etuk notes, if they are large, a t-test may be fine. And if they are equal, a t-test will be very robust to heterogeneity of variance.)
- What is the research question you are trying to answer?
Finally, note that the Wilcoxon-Mann-Whitney test is quite sensitive to small differences in variance and skewness--see the nice simulation study by Fagerland & Sandvik, link given below. This is important if you are using it as a test of location.
Article The Wilcoxon-Mann-Whitney test under scrutiny
HTH.
@ManjulaWijewickrema,
Effect size is not a test.
For an ordinal dependent variable and a categorical independent variable (the Mann-Whitney case), appropriate effect sizes are Freeman's theta and epsilon-squared. Because you have only two groups, you could also use Kendall's tau as an effect size.
For a continuous dependent variable and a categorical independent variable (the t-test case), appropriate effect size is Cohen's d.
The classic text for interpreting effect sizes is Cohen, "Statistical Power Analysis for the Behavioral Sciences". But the interpretation of effect sizes depends greatly upon the discipline.
Also, note that t-test, M-W, and K-S all test for different things. (Look up their null hypotheses).
And their corresponding effect sizes express different things.
You may want to start with asking yourself what it is you are trying to determine about these two different variables.
@Bruce,
The final conclusion of the paper you link to is "The ... [Mann-Whitney] test can be a poor method for comparing the means or medians of two populations, unless the two distributions have equal shapes and equal scales." This stems from the fact that their simulations looked at the rejection rates for null hypotheses dealing with the means and medians.
But it should be noted that the hypothesis for the M-W test isn't about means or medians. So it seems like an unfair condemnation of the test when we should be condemning the users of the test who aren't getting correct the null hypothesis that the test considers. Blaming the shoes for faults of the feet, so to speak.
Hi Salvatore. I agree with what you've said about the null hypothesis for the Wilcoxon-Mann-Whitney (WMW) test, and so do Fagerland & Sandvik. But I daresay that many (if not most) users of the WMW test continue to believe it is testing a null that is exclusively about location (e.g., the population medians are the same), regardless of the shapes of the underlying populations. The point of the simulations is to show readers (who may find simulation results more digestible than mathematical equations) that this just is not so.
Cheers,
Bruce
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Physical Phenomena
- Luminescence
- Fluorescence