When using a Wilcoxon signed rank test pairs of data that have a difference score of zero are removed from the analysis and the sample size reduces. How then should the sample size be reported? For example if you have 159 participants, but in the analysis 13 pairs of data are removed do you report that you had n=159 or n=146?
Secondly, if this test essentially excludes participants where no change has occurred, is this predisposing the analysis towards finding a difference?
The signed rank test is for continuous data, and this is because then Pr(xi==xj) = 0 for any i, j in (1,n) and i ≠ j. Scores are typically not continuous and therefore rank tests are actually unsuited for this kind of data (although the opposite is often trumpeted in many textbooks, mostly for reasons related to the distributional assumptions that would prohibit using a t-test). In some circumstances, it may work, and than one might be "happy" (using a wrong tool for the task that makes not so much sense for the data).
If you want to estimate the stochastic equivalence or the mean or median difference, get a confidence interval and/or a p-value for some hypothesis, the only "correct" way is to bootstrap the sampling distribution from your data. With n=159 there should be a good chance to get reasonable results.
However, this does not mean that other approximate procedures would be forbidden. With n=159 you do have a relatively large sample, so large-sample approximations for the mean difference will likely work acceptably well (-> paired t-test). Also, the results from the signed rank test after removing the ties (if there are not too many of them) will given a sufficiently accurate test of the stochastic equivalence.
If you stay with the signed rank test, you should make clear that the test is approximate and how many ties were in the data (159-146=13; I think 8% is still ok). You may check what the most extreme impact of the ties can be by substituting all zero-differences by +1 or by -1, and compare the results. Are the differences between these results relevant for you?
The signed rank test is for continuous data, and this is because then Pr(xi==xj) = 0 for any i, j in (1,n) and i ≠ j. Scores are typically not continuous and therefore rank tests are actually unsuited for this kind of data (although the opposite is often trumpeted in many textbooks, mostly for reasons related to the distributional assumptions that would prohibit using a t-test). In some circumstances, it may work, and than one might be "happy" (using a wrong tool for the task that makes not so much sense for the data).
If you want to estimate the stochastic equivalence or the mean or median difference, get a confidence interval and/or a p-value for some hypothesis, the only "correct" way is to bootstrap the sampling distribution from your data. With n=159 there should be a good chance to get reasonable results.
However, this does not mean that other approximate procedures would be forbidden. With n=159 you do have a relatively large sample, so large-sample approximations for the mean difference will likely work acceptably well (-> paired t-test). Also, the results from the signed rank test after removing the ties (if there are not too many of them) will given a sufficiently accurate test of the stochastic equivalence.
If you stay with the signed rank test, you should make clear that the test is approximate and how many ties were in the data (159-146=13; I think 8% is still ok). You may check what the most extreme impact of the ties can be by substituting all zero-differences by +1 or by -1, and compare the results. Are the differences between these results relevant for you?
Thankyou for this comprehensive reply.
The data I'm dealing with is a 5 point scale and I was intending to use the signed rank test because that's what others in the literature had used. Given that this test, and the t-test as well are for continuous data I think neither is a great fit.
Now I'm thinking McNemar's test would be better - treat this like categorical data and use the test to look at category shift, or not. This would be a far better way to capture those for whom there is no change.
Does this sound like a feasible plan?
Could be. Another option is an ordered logistic model. However, I would either consider a large sample approximation or adress the relevant question directly using a randomization/bootstrap technique.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size