If the similarity of shape assumption is violated with a very short sample size, consider using non-parametric tests that do not assume equal variances or shapes, such as the Mann-Whitney U test for comparing the ranks of two independent groups. Additionally, ensure the robustness of results by conducting sensitivity analyses or bootstrapping methods.

In case the data is not normally distributed and if your goal is to compare the mean rank of two groups, a Kruskall-Wallis test is a better option. I have a video of it on my YouTube channel on it. https://www.youtube.com/watch?v=1vf7YXcHoEk.

Hello Babita Yadav. You many find this BMJ Statistics Note helpful.

Bland JM, Altman DG. Analysis of continuous data from small samples. BMJ. 2009 Apr 6;338:a3166. doi: 10.1136/bmj.a3166. PMID: 19349342. https://www.bmj.com/content/338/bmj.a3166.long

Those are fair comments, Jochen Wilhelm

The aversion to parametric methods for small samples may arise from the inability to assess the distribution shape when there are so few observations. How can we tell whether data follow a normal distribution if we have only a few observations? The answer is that we have not only the data to be analysed, but usually also experience of other sets of measurements of the same thing. In addition, general experience tells us that body size measurements are usually approximately normal, as are the logarithms of many blood concentrations and the square roots of counts.

PS- Something went wrong on two previous attempts to post this. Apologies if multiple versions of the post show up eventually. If that happens, I'll delete some of them.

The Wilcoxon-Mann-Whitney test doesn't really require that the distributions have the same shape.

Sometimes this assumption is added by textbook authors to simply things.

[ Personally, I find this a little silly. Why take a perfectly good nonparametric test and start imposing restrictions on the distributions ?]

But without that assumption, it can get a little tricky to identify the assumptions and hypotheses of the test.

Conover, Practical Nonparametric Statistics, tackles this by making the hypotheses:

H0: All of the k population distributions functions are identical.

H1: At least one of the populations tends to yield larger observations than at least one of the other populations.

This just avoids the case when the distributions are stochastically equal but aren't identical. So you could say the assumption of the test in this view is that either the distributions are identical or one tends to yield larger observations than the other.

Practically, I think this is fine i most cases.

I have a longer response here, https://www.researchgate.net/post/I_have_non-normal_data_with_heterogeneity_of_variance_which_ANOVA-type_procedure_and_post_hoc_tests_should_I_use .

Hi, Jochen Wilhelm

Definitely some authors use the "same shape and spread" assumption to make the test a test of location shift. Some --- thankfully --- have a separate set of less-restrictive assumptions for the more general case.

But you can see by the odd hypotheses Conover uses, that something else is going on here.

My assumption --- and I don't think I ever confirmed this --- is that if you have a case of stochastic equality but different variances in the populations, that the type-I error rate would be a bit inflated. I assume this is just because in the case of the higher variance there is a chance that you happen to sample, say, a bunch of the higher values.

You see this with permutation tests like Fisher-Pitman, which --- to my understanding --- can also identify differences in variance.

I think in practice this isn't something to usually worry about. But maybe in some cases...

What to do?

my honest answer - just plot two groups. You'll see what happens with Your data

But discussion is really very useful

Olga Krasko I like your answer. If significance testing can be avoided, I would go with that. If not, what do we have here? The dilemma, in my view, is that the MW U tests the null hypothesis whether the two samples have the same distribution. But there is already an understanding that one is normal while the other is skewed. Given the small samples and the additional layer introduced by the hypothesis, I would be a bit concerned to draw a strong inference. Thus, I would ask whether the gender and the AT can be linked. I would use gender and the median rank (above and below median) to count the children falling within each cell of a 2 by 2 table and do a Fisher's Exact Chi Square test.

Darshi. Arachige Thanks!

We still can do ROC -analysis too.

I'd like to partially disagree with a couple of the previous comments. Hypothesis tests shouldn't be avoided. It's just that the p-value doesn't tell you everything.

It's fine to use the Wilcoxon-Mann-Whitney (Mann-Whitney U) test. This is a test about whether the observations in one group tend to be larger than in the other group. It's not really a test if the "distributions are the same". And I wouldn't switch to the median test, unless this is actually the hypothesis of interest.

In this specific case, I think the issue is that the sample size is really rather small. However, if the samples have limited overlap in terms of magnitude of the observations, you may get a significant p-value.

I think a plot would be helpful. With these few data, a bee swarm plot showing all the data is useful.

Also, a standardized effect size statistic is useful to indicate the proportion of observations that are larger in one group than the other. There a few different ones that are used, all of which convey similar information. These include probability of superiority, Glass rank biserial correlation, and Vargha and Delaney's A.

Here is an example with some data I invented.

A = c(1, 3, 5, 7)

B = c(4, 6, 8, 8.1, 8.2, 8.3, 8.4)

wilcox.test(A, B, correct=FALSE)

### p-value = 0.04242

Y = c(A, B)

Group = c(rep("A", length(A)), rep("B", length(B)))

library(rcompanion)

wilcoxonRG(Y, Group)

### rg

### -0.786

### This is the Glass rank biserial correlation. It's "scale" is like Pearson's r, ranging from -1 to 1.

wilcoxonPS(x=A, y=B, verbose=TRUE)

### Statistic Value

### Proportion Ya > Yb 0.107

### Proportion Ya < Yb 0.893

### Proportion ties 0.000

###

### PS

### 0.107

### This is the probability of superiority.

Sal Mangiafico Thank you very much for your detailed response about my suggestion. Appreciated! The remark about the same distributions, rightly or wrongly, was based on my interpretation of M & W's original claim that f and g are continuous cumulative distribution functions used in testing the hypothesis f=g. I am sorry if my answer anyhow reflected a dislike to use hypothesis tests. The answer was specific to the question at hand and included a hypothesis test with a more relaxed null hypothesis. The problem I find in some two sample significance test situations can also be seen in your example data. Two samples A & B shows a 40% overlap (Technical Report Some Practical Considerations about the t-Test for Two Indep...

Hi, Darshi. Arachige .

As to what the WMW tests for, it is (usually) a test of whether one group has higher values than the other group. It isn't a test of "same distribution", as if the two samples have very different distributions, but are stochastically equal, the p-value will be precisely 1. A simple example follows.

J = c(1,1,1,5,5,5)

K = c(3,3,3,3,3,3)

wilcox.test(J,K, correct=FALSE)

### p-value = 1

It's fine to use overlap as an effect size statistic. However, an overlap of 50% to me suggests two very different samples.

I don't know if "overlap" here is calculated the same way you're calculating it, but it should be in the same ballpark (I assume). But here is an example of two groups, 50% overlap, with Cohen's d: https://imgur.com/a/RJ2HbbD (from https://rpsychologist.com/cohend/ ).

To me, those are two interestingly different groups.

Also note the probability of superiority of 83%. That's not far off from the probability of superiority in my A, B example above. That means the probability of an observation in Group 2 being higher than an observation in Group 1 is 83%. To me that's pretty striking.

Sal Mangiafico Thank you for the explanation, Prof. Mangiafico. Please note that my point was not about the way the test is used, but one of the underlying assumptions, as stated in the M&W's paper, that, to my understanding, the test was built around.

Cohen's d I think is about the difference between means standardised by the pooled SDs. My main concern originally arose from the dispersion of the samples drawn (i.e. realised values). Based on the possible ranges, my calculation with your data was only an estimate.

Hi, @Darshi. Arachige

I'm a little lost as to the point you're trying to make.

It's good to look at the original paper, but it's also important to know how a method has been expanded or re-examined since then. ( The paper was 77 years ago ! )

If the point is about the variable being continuous... A lot of nonparametric were originally designed for continuous variables. But if an implementation can handle ties correctly, some work fine for ordinal data. Here is some results of a simulation I did to compare WMW to ordinal regression on 5-point Likert type data. Not definitive, but suggestive. https://rcompanion.org/handbook/images/image304.png

If the point is about the f(x) = g(x) hypothesis... I think Mann and Whitney stated the hypotheses well: "We wish to test the hypothesis f = g against the alternative that x is stochastically smaller than y."

The alternative is important here, and this is how Conover lists the hypotheses, as I quoted above. If the sample distributions are different, but stochastically equal, the p-value will be 1.