In my experiment I have two groups: a clinical group (n=15) and a control (n=15).
I have used SPSS to analyse the data.
Dependent variable 1 was normally distributed, so group differences on this variable were analysed using an independent t test. I calculated the effect size for dependent variable 1using cohens d.
Dependent variable 2 was not normally distributed, so group differences on this variable were analysed using a Mann Whitney U test. I calculated the effect size for variable 2 using Eta Squared.
I have been asked to provide confidence intervals for my effect sizes. But I have a few queries about how to do this:
Qu1. For Variable 1 (for which I used a t-test & cohens d) do I just provide the confidence intervals from the SPSS independent t test output table?
Qu2. For Variable 2, am I correct in thinking that Confidence Intervals are not typically provided for non parametric tests?
Qu3. How can I calculate the confidence intervals for variable 2 Mann Whitney U/eta squared?
(I have read about the Hodges-Lehmann estimation, but as a psychologist with limited statistical understanding , I am not able to calculate this, so any other solutions or advice would be much appreciated).
Thanks in advance for any help with this,
Best wishes,
Iona
Hello Iona,
Either of these links will give you the tools you need:
1. http://daniellakens.blogspot.com/2014/06/calculating-confidence-intervals-for.html
2. https://thenewstatistics.com/itns/esci/
Good luck with your work!
Thanks again David Morse for your help with this, much appreciated, I will give these tools a try.
Best wishes,
Iona
There's no reason why you can't have a confidence interval for a nonparametric statistic. My guess is that you will need to use bootstrapping to develop the confidence intervals for the statistic of interest. In any case, be sure you are using the eta-squared statistic that is appropriate for Mann-Whitney (see Tomczak and Tomczak, http://tss.awf.poznan.pl/files/3_Trends_Vol21_2014__no1_20.pdf ). There's also an eta-squared that is used for parametric analysis of variance (anova), so you need to be careful you have the right one. In my next response, I'll give a potential solution.
Because you have only 30 values to analyze, it's not too difficult to copy the data into different software.
One difficulty I'm having in trying to answer this, is that eta squared for Mann Whitney is defined a couple of different ways, and the results are slightly different. (See Tomczak and Tomczak and the discussion here: https://www.researchgate.net/post/Anyone_know_how_to_calculate_eta_squared_for_a_Kruskal-Wallis_analysis )
So I'll present the simplest solution. The following is for a two-sample problem (Mann-Whitney) only.
In R, there is a function in the rcompanion package, WilcoxonR, that calculates r for Mann-Whitney. (Caveat: I am the author of this package). I'll just square this value as suggested by Philippe Jawinski in the link above. The following code can be run at https://rdrr.io/snippets/ without installing R. You just need to change the values in A and B.
If you use this code, please compare to the result for eta squared you got previously. It might be slightly different, for a variety of reasons. I can offer some different solutions if it's important to you to match the SPSS results.
For this example, because r is negative (group B's values are larger), the upper and lower bounds for etaSquared are switched when the r values are squared.
A = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
B = c(5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19)
Values = c(A, B)
Group = factor(c(rep("A", length(A)), rep ("B", length(B))))
library(rcompanion)
R = wilcoxonR(x=Values, g=Group, ci=TRUE)
R
etaSquared = R ^ 2
names(etaSquared) = c("eta.squared", "lower.ci", "upper.ci")
etaSquared
Sal Mangiafico thank you for your detailed guidance, I will certainly have a look. Your advice is much appreciated.
Best wishes,
Iona
As a follow-up, you might consider an effect size that's easier to interpret. Vargha and Delaney's A has a simple interpretation: the probability of a value from group being larger than a value than the other group. ( There's also a linear translation of this statistic called Cliff's delta.) This makes perfect sense for a Mann-Whitney test. More code below. Note that for VDA, 0.50 is no effects and 0 and 1 indicate maximum effect. (These are probability values.) For Cliff's delta, 0 is no effect, and -1 and 1 indicate maximum effect.
A = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
B = c(5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19)
library(rcompanion)
vda(x = A, y = B, ci=TRUE)
cliffDelta(x = A, y = B, ci=TRUE)
Stata users can get the probability that a randomly selected observation from one group is greater than a randomly selected observation from the other group by adding the porder option to the -ranksum- command. There is a nice example in this Stata Journal article by Ronán Michael Conroy. HTH.
- https://www.stata-journal.com/article.html?article=st0253
Thanks Bruce Weaver for the suggestion- Stata sounds like a good program. Unfortunately I only use SPSS.
Thanks for the advice,
Much appreciated, Iona
Iona Walker – you poor thing!
Never fear, the probability that Bruce Weaver mentions is easy to calculate from the value of U. It's U/(N1*N2), that is, U divided by its maximum value. Sadly, Mann and Whitney forgot to put this in their paper, despite the fact that the title promised exactly that!
So with SPSS and the calculator on your phone, you too can have a measure of effect size.
Ronán Michael Conroy, thanks for that reminder about U/(N1*N2). I need to add that to my lecture slides for the benefit of SPSS users.
Thanks Ronán Michael Conroy for the formula for spss users and For taking the time to answer.
best wishes,
Iona
It's really silly – Mann and Whitney's paper was
Mann HB, Whitney DR. On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. Ann Math Statist. 1947 Jan 1;18(1):50–60.
The title says it all. They had in mind the probability that we've been talking about, but in the paper they stopped just one step short of calculating that probability! For goodness sake, one more formula was all they needed.
And as a result, at least half a dozen people have proudly announced that they have invented that very statistic.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA