Many statistics, like z and t ---- as well as many effect size statistics, like r, Cohen's d, and Cliff's delta ---- are signed. Typically, with these statistics that compare two groups, a negative sign means that the second group has larger values than the first group. Obviously, which group is the first group is relative to how the groups were coded and how the software decides which group comes first. So, it's common and informative to report positive or negative z-values, t-values, or r-values.

I don't know what "z score is 'based on negative ranks' " would mean. I'm also wondering about the fact that your post-intervention had a lower Mdn than did the baseline, but that you said that the post-intervention had higher values.

Thank you for replying, I put the medians in the wrong way round (I'm going cross-eyed with all this stuff), I have changed them to read correctly.

z score is "based on negative ranks" is stated at the bottom of the table in SPSS

So if the r = -0.48, would that be reported as a medium effect size?

Hello Patrina,

The magnitude/import of an observed effect size is in the eye of the beholder. If you want to cite Jacob Cohen's guidelines (in the absence of any other context...though Cohen always noted that context matters), then his thresholds for "small," "medium," and "large" correlation coefficients (r) were .1, .3, and .5, respectively (disregarding sign).

Good luck with your work.

That's great, thank you so much! Would I have to provide a reference for disregarding the sign?

Hi, Patrina Bevan . You shouldn't need a reference for a signed effect size statistic.

A question: You say you used Wilcoxon signed rank test. Did you actually use Wilcoxon ranked-sum test instead?

Regarding Sal Mangiafico's question in his most recent post, please share with us the command syntax. Even if you are a point-and-click user, the command syntax should appear in the output window. Thanks.

With the Wilcoxon signed ranks test one computes for each pair of scores a signed difference score. Some may be positive and some may be negative. Then, ignoring the signs (that is, using absolute values) the scores are ranked. Then the signs are reattached to the ranks. Two sums of then calculated -- the sum of the negatively signed ranks and the sum of the positively signed ranks. The smaller of those two sums is the test statistic. Whether the smaller sum is a negative sum or a positive sum is an artifact of whether you subtracted Condition 1 from Condition 2 or Condition 2 from Condition 1.

See https://core.ecu.edu/wuenschk/docs30/Nonparametric-EffectSize.pdf for alternative effect size estimators.

Karl Wuensch , thank you for linking to that document. It is helpful.

I wanted to mention ---- that for either the paired or unpaired case ---- the r that is z divided sqrt(N) ---- while relatively common ---- isn't the best-behaved effect size statistic.

I recommend statistics that are either better-behaved or have intuitive interpretations. Like rank biserial correlation coefficient, matched-pairs rank biserial correlation coefficient, common language effect size statistic, Cliff's delta, or Vargha and Delaney's A. Some of these are also described in the Wikipedia page (https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test).

With the caveat that I wrote the functions, I have methods for these effect size statistics in the R package rcompanion (https://cran.r-project.org/package=rcompanion). Confidence intervals by bootstrap can also be requested. For those who don't use R, these functions are easy to run online at https://rdrr.io/snippets/ . For example,

BartSimpson = c(10, 11, 12, 10, 11, 9, 8, 13, 14)

LisaSimpson = c(21, 18, 14, 12, 12, 11, 19, 22, 23)

wilcox.test(BartSimpson, LisaSimpson)

library(rcompanion)

vda(x = BartSimpson, y = LisaSimpson)

cliffDelta(x = BartSimpson, y = LisaSimpson)

Y = c(BartSimpson, LisaSimpson)

Group =c(rep("Bart", length(BartSimpson)), rep("Lisa", length(LisaSimpson)))

wilcoxonRG(Y, Group)

### Compare to r = z / sqrt(N)

wilcoxonR(Y, Group)

This is the syntax

DATASET ACTIVATE DataSet2.

NPAR TESTS

/WILCOXON=B_WEMWBS WITH S_WEMWBS (PAIRED)

/MISSING ANALYSIS.

I have been using SPSS and calculated effect size by

the r that is z divided sqrt(N) where (N) is the total number of observations

I do hope this is ok...I don't think my head could take it if it isn't!

Patrina Bevan ,

1) Well, the first question I would ask is, how can you have a paired analysis if you have a different number of observations in each group ?

2) In terms of calculating r, in the paired case, N should be the number of pairs. ... There's some disagreement about this, but I really can't fathom the argument that N should be twice the number of pairs. a) This really doesn't make sense considering how the test is conducted. (The test is performed on the difference in pairs). b) The results make less sense: b1) The paired sample test is the same as a one-sample test on the differences in pairs. For the effect size statistic to be the same, N needs to be the number of pairs. b2) r should be close to 1 when all the observations of one group are greater than their paired observations, as in the following case. Here, I get an r of 0.89 when N is the number of pairs. If N is twice the number of pairs, r is only 0.63.

A = (11,12,13,14,15,16,17,18)

B = (22,24,26,28,30,32,34,36)

c) Note that in the above case, the matched-pairs rank biserial correlation coefficient will return -1, while r returns -0.89. This is an example of a deficiency with this r statistic.

Sample R code:

A = c(11,12,13,14,15,16,17,18)

B = c(22,24,26,28,30,32,34,36)

wilcox.test(A, B, paired=TRUE)

### Wilcoxon signed rank exact test

### V = 0, p-value = 0.007813

Y = c(A,B)

Group = c(rep("A", length(A)), rep("B", length(B)))

library(rcompanion)

wilcoxonPairedR(Y, Group)

### r

### -0.891

wilcoxonPairedRC(Y, Group)

### rc

### -1

scores XX were significantly higher at post-intervention (Mdn = 53.00, n = 25) compared to baseline condition (Mdn = 43.00, n = 27),

My guess is that that there were two missing data points post-intervention and that the baseline median should have been calculated using only the 25 cases also had post-intervention scores. SPSS NPar, when you ask for descriptive stats, gives means, not medians.

It is usually to use the absolute value of z score , it is the first time for me to see this.. please let me updated

Sal Mangiafico Karl Wuensch Khaled Elnagar David Morse

Thank you so much for taking the time to help me with this, as you can see I'm a complete novice to statistics!

I had an online survey that consisted of three questionnaires, which were taken at three timepoints (baseline, post-intervention, and follow-up).

The first questionnaire was completed 25 times at baseline, post-intervention, and follow-up (although not by the same 25 people)

The second questionnaire was completed 27 times at baseline, 26 times at post-intervention, and 25 times at follow-up (not necessarily by the same people)

The third questionnaire was completed 18 times at baseline,17 times @ post-intervention, and 17 times @ follow-up (not necessarily by the same people).

I ran the Wilcoxon signed-rank for baseline and post-intervention as it has the greatest number of responses.

Should I run a different test for unrelated samples?

Only 11 people completed the full survey (all three questionnaires) at all three timepoints. So these are my related samples, correct?

For this sample I ran a Friedmans then, to see where the differences lay, a Wilcoxon signed-rank.

This is one tutorial that says to calculate the effect size, the square root of the number of observations is used, not the number of pairs.

https://www.youtube.com/watch?v=1zbpiNY2YWE&t=532s

This is one of the websites I accessed.

https://statistics.laerd.com/spss-tutorials/wilcoxon-signed-rank-test-using-spss-statistics.php

Thank you again so much for your help, it's very much appreciated :)

Patrina Bevan ,

• I would say that that video is incorrect about calculating the effect size statistic r. See my arguments above. I mean, anyone can calculate any statistic they want and call it whatever they want. But with this r statistic with the signed rank test, using the number of pairs makes sense, and using the total observation doesn't really make sense. ... (Although as I mentioned, this r statistic isn't the best effect size statistic. The matched-pairs rank biserial correlation coefficient is just better.)
• Not to be too contrary, but I also have some disagreement with the Laerd page also. In particular their Assumption #3. Also, their contention that this test works with ordinal data.
• "not necessarily by the same people" makes this analysis complicated. ... To be clear, if you are using a test for matched data ---- signed-rank or Friedman ---- you can only use those observations that can be matched (in this case, by being an observation of the same person). There's just no way to use these tests if some observations are unmatched.
• So, it may make sense to 1) Discard unmatched observations and use a test for paired data. Or, 2) Ignore the fact that some observations are paired, use tests for unmatched observations, and just note in your results that some of the responses are from the same person. ... From your description, I don't know which might be a better approach.
• A Wilcoxon signed rank test isn't a great match as a post-hoc test for Friedman. The tests use different approaches. There are appropriate post-hoc tests for Friedman. Likewise, there are tests that are more similar to the signed rank approach. Unfortunately, we get limited by what tests and statistics are available in SPSS.

Sal Mangiafico thank you so much for responding :)

You’ve convinced me that effect size should be the number of pairs and not the number of observations 😊

If I ignore the fact that some observations are paired, what tests should I use for unmatched observations, please? And how would I calculate the effect size for that? Would it be ok to use more observations for one timepoint than another (as above)?

For the matched data (11 participants at 3 timepoints), do you think it is ok to use the Friedman? Would you then suggest the pairwise comparison test to show where the differences lie? How would I calculate effect size?

Oh! How I wish I had a brain that understood statistics!

Thank you

Hi, Patrina Bevan .

Using classic non-parametric statistics in particular can be tricky. One factor is that the tests themselves can validly be viewed as relying on different assumptions or have results that have different interpretations. Another factor is that a lot of information on websites (or even textbooks !) aren't as clear, or sometimes as correct (!), as they should be.

You definitely get bonus points for using an effect size statistic.

I'm getting a little confused as to the design of the study. But, yes, Friedman test might be an appropriate for the matched data. Quade test might be preferable in this case. ( I think this is not offered in SPSS, though there are procedures online.)

Kruskal-Wallis would be a nonparametric test for multiple groups.

I don't know that I would recommend analyzing some of the data as matched, and some of the data as unmatched. I'm not sure. It might be better to treat all the data as unmatched and simply note that in some cases the responses were from the same individual. Otherwise, you have only 11 (?) observations that are matched ?

Sal Mangiafico thank you, once again.

I have decided not to analyse the non-matched data. I have 20 participants who completed at baseline at post-intervention (including the 11 who did all three timepoints), so I think I will just do those.

Sounds reasonable. If you have only two time points, I guess that brings you back to the signed rank test ...