Hello Alan,

For instances in which the Mann-Whitney U is an appropriate vehicle for comparing equality of two sets of (ordinal or higher strength) scores, if the difference does not meet your criterion for statistical significance/dependability, then there is no discernible difference between the sets/groups. In other words, there's no statistical evidence to assert that scores were higher in one group (compared to the other).

If you'd prefer to use a statistic that directly addresses the potential for one group having higher scores than another, either of these might be useful:

1. Common language effect size (CLES): How frequently (the probability that) a randomly selected case from group A exceeds the score of a randomly selected case from group B (ignoring tied cases). This can be a strict, nonparametric test statistic.

2. Norman Cliff's (1996) proportion dominance statistic. Another nonparametric statistic. Quantifies proportion of times randomly selected cases yield (a) A > B; subtracting instances in which (b) B > A. So, a positive value suggests A cases generally have higher scores than B cases; a negative score suggests the opposite; zero scores suggest no difference.

Good luck with your work!

References:

Cliff, N. (1996).  Ordinal methods for behavioral data analysis.  Mahwah, NJ: Lawrence Erlbaum.

McGraw, K. O., & Wong, Sp. P. (1992).  A common language effect size statistic.  Psychological Bulletin, 111, 361-365.

If you want to make a statement about estimates (or differences between estimates) you will need Bayesian statsitics. You need to specify a sensible prior (that encodes what you know or would claim about the estimates before the experiment) and use the experimental data to modify the given priors, what results in your posterior (that encodes what you know or would claim about the estimates after knowing the data). You can then report the posterior distribution of the estimate (or the difference), maybe summarized by highest posterior density intervals (HPDs).

Sounds like you have already done the test and found no significant difference between athletes and controls. Assuming you have done the best test, then the conclusion must be: these data provide no evidence to reject the null hypothesis that athlete and control CTX scores differ.

So you cannot claim that athletes and controls differ in any way,

Now the two groups that you are sampling from may differ in a biologically meaningful way but your sample may be too small to detect the difference. That just means that the power of your test is too low and that collecting more observation is needed. But it could mean that there is genuinely no difference. You need to ask if you have done the most powerful snd appropriate test for your data.

Novel idea: Use descriptive statistics and dive deep into the study limitations in your paper. Use what you learned to inform your statistical analysis and study design for your next study.

Just a thought, most studies are not groundbreaking in their findings. Research is a stepwise process of discovery.

Regards,

Christopher

All statistical analysis should be 90% exploratory and descriptive work before testing for significant differences. You should know your data in and out to effectively test your hypothesis with the proper model. The testing should take only a few minutes if it is based on a good knowledge of the structure of the data and if the data is clean.

The interpretation will take longer but careful checking of your results and proper interpretation can and should take as much time as needed.

In this case, I would use your present data to estimate the sample size needed to obtain at least 80% power in your next text.

You can calculate the effect size from the output to your test. The probability that a person in the higher-scoring group will have a higher score than a person in the lower-scoring group is U/(N1*N2) where U is the test statistic and N1 and N2 are the two sample sizes.