You can go with Mann-Whitney U test alone.

If your hypothesis states that productivity in both groups is the same, then you need to define same. Clearly, it's not going to be exactly the same, so you need to establish a margin around zero that you define as 'same'. This is the principle behind equivalence testing. So the Wilcoxon Mann-Whitney test isn't going to help here unless you use it to calculate Harrell's C and its confidence interval.

The second problem you have is that your dependent variable as counted, and is bounded by zero and the number of tests. Using negative binomial regression would allow you to express the comparative performance of the two groups using an incidence rate ratio. You could set your definition of 'same' as, for example, 'no more than ±10% difference in rate of solving. Your incidence rate ratio's confidence interval, then, must be inside the region 0·9 to 1·1.

Thank you very much for your help, Mr. Conroy.

What if my hypothesis stated that productivity was different? I have seen many researchers in the field of economics who have firstly utilized a Mann-Whitney U-test and secondly applied a linear regression to confirm results. (e.g. Georganas et al., 2015; Falk and Ichino, 2006)

Does such an approach even make sense? Why not checking for normal distribution in the first place and using a t-test if appropriate? Afterwards one may implement a linear regression to integrate control variables.

Why is a Mann-Whitney U-test always applied in the first place?

Kind regards and thanks for further help.

Georganas, S., Tonin, M., and Vlassopoulos, M. (2015). Peer pressure and productivity: the role of observing and being observed. In: Journal of Economic Behavior and Organization, 117, 223–232.

Falk, A., and Ichino, A. (2006). Clean evidence on peer effects. In: Journal of Labor Economics, 24 (1), 39–57.

Again, you have to choose between "different" meaning "statistically significantly different" and different meaning different enough to be meaningful. For example, routine exercise ECG testing has the benefit of an extra half day of life. Of course, you waste that extra half day getting the exercise test.

So you need to define what would constitute a real-life difference between the groups and set your sample size to have a good (90%+) power of detecting this.

Dear Mr. Conroy,

I think there was a slight misunderstanding regarding my updated comment:

my updated comment was in case that I was testing for different means and not for same means. So in case that I was testing H0: µ1= µ2 ; H1: µ1 > µ2 (so not testing for equivalency anymore).

Does it make sense to firstly utilize a Mann-Whitney U-test and secondly apply a linear regression to confirm results? Why not checking for normal distribution in the first place and using a t-test if appropriate? Afterwards one may implement a linear regression to integrate control variables.

Thank you very much for your help and time.

Linear regression tests for a difference in means. Wilcoxon Mann-Whitney does not test for a difference in any descriptive statistic. It tests the hypothesis that an observation from one group will be larger than an observation from the other group. So it can be significant when there is no difference in the means, and vice versa.

Why linear regression anyway? Your data are numbers of successes out of a fixed number of trials.

I think, Sebastain Huber is asking that, is it ok to apply linear regression for cause and effect after applying Mann Whitney test for the significant difference as we apply linear regression easily after an independent t-test?

I also have little confusion on this. Mann Whitney test is non-parametric test while linear regression is a parametric one. If not are there any options (conditions and assumptions) to apply regression after a non- parametric test ?

Deepak Raj Joshi the Wilcoxon Mann-Whitney is not a nonparmetric test because it estimates a useful population parameter – the probability that an observation from one group will be bigger than an observation from the other group. This is useful in estimating, for example, the chances that a patient will do better on one treatment than on another, or that one product will last longer than another.

For these data I'd consider a count model such as Poisson regression (or probably better still negative binomial regression to allow for overdispersion). You could run incorporate the baseline as a covariate if necessary.

Ronán Michael Conroy thank you for your reply. So can we use regression after testing difference from Mann Whitney U test? And another thing is we have studied Mann Whitney U test as a distribution free test (non-parametric) test and used in place of independent T-test. Could you please suggest any reference for this? Will be grateful for this.

Deepak Raj Joshi – whoa! The WIlcoxon Mann-Whitney and regression test different hypotheses. The most basic form of OLS regression is the t-test, and if the grouping variable is coded with consecutive integer values (say 0 and 1) then the coefficient is the difference between the means.

So regression is not an extension of the Wilcoxon Mann-Whitney. Indeed, you can find a significant effect for one hypothesis test and not the other.

I've attached, for general edification, maternal BMI values for two groups of mothers. If you run a t-test the difference in means isn't significant (and looking at the means you can see why) but the WIlcoxon Mann-Whitney correctly tells you that a mother whose baby was delivered by emergency section is 65% likely to have a higher BMI than a mother whose baby was delivered normally.

Ronán Michael Conroy Thank you so much !!

We proposed a new dependence measure between categorical and numerical variables and consequently a new test on distribution equality among groups can be performed to test the new correlation zero. The new test has many nice properties such as consistency and high power, especially in unbalance cases. Ronán Michael Conroy

Dang, X., Nguyen, D., Chen, Y. and Zhang, J. (2021). A New Gini Correlation between Quantitative and Qualitative Variables. Scandinavian Journal of Statistics, 48 (4), 1314-1343. https://doi.org/10.1111/sjos.12490

Xin Dang – a very interesting approach. How would you explain what the correlation measures if you were using it in a paper for, say, health professionals? How would you interpret a given value in real-life terms?

I ask this because I fear that without an intelligible interpretation, apparently precise quantitative measures become understood qualitatively – often by creating "tee-shirt sizes". You can think of "small" "medium" and "large" kappas, or alphas or d values.

Ronán Michael Conroy - I really appreciate the question which puts me in the shoe of health professionals. In real-life, alpha is set to be 1 for computation ease and simple interpretation on the euclidian distances used in the correlation. d is the dimension of the numerical variable(s). So the method is also good for comparison of several variables together among K samples.

About "small", "medium", and "large" correlation values really depend on underlying assumptions on distributions of each class. Take the balanced two groups for example. Group 1 is standard normally distributed N(0,1); while Group 2 is from N(a,1) with a>0. The difference of two groups is location. The larger a is, the more separable of two groups. For a = 1, 2, 3, 4 and 5 which correspond to the Bayes classification errors (the smallest error possible for classification) to be 0.31, 0.16, 0.065, 0.025 and 0.005, the proposed Gini correlations are 0.11, 0.30, 0.46, 0.56 and 0.63, respectively. Hence I would say in this case, 0.1 is weak, 0.3-0.45 is medium, >0.45 is strong.

Without knowing the underlying distributions, the p-value of the test provides more information than the value of the correlation itself.