Hello All,
I can see sometimes The H0 null hypothesis has been rejected using Wilcoxson-sign rank test, in this case I was expected to confidence interval not include zero but it includes. why?
for instance:
median_diff : -3
Lower limit : -12
Upper Limit : 1
Test statistic : -40.5
p value : 0.0385
The signed rank test isn't a test of the median. ... Are these results all coming from the same software function ?
Sal Mangiafico The Wilcoxon signed rank test checks if the median difference between paired observations = 0, hence the output.
Bahar Ysr This result is pretty fun, because it is as you say a contradiction. The CI computation for the median difference involves sample quantiles/order statistics, whereas the p-value from the test is based on the distribution of the ranks of the differences. The signed rank test is sensitive to the sign of differences (+magnitude of the ranks), while the CI focuses on a central range around the median difference. In this case, your significant p value indicates that the ranks of the differences are significantly skewed in one direction -> reject H0, yet the CI∈[-12, 1].
So while the rank test identified a statistically significant result (based on ranks), the CI shows that the estimate of the median difference is imprecise (reflected by the large range), and there's still some chance the true median difference could be zero. This happened in your case, and usually only happen in cases with smaller sample sizes with a lot of variability in the data.
Thanks so much for your help; it was helpful; I was wondering if it is correct to say:
These confidence intervals are distribution-free, so they may not have exactly 95% coverage, but here, they have at least 95% coverage. For all the median Confidence Intervals, the coverage is ~96%, so you may see 0 included in the confidence intervals when the p-value is less than 0.05.
Thanks
Marius Ole Johansen . No. The signed rank test is not a test of the median difference.
It's easy to come up with toy data where the median of differences is 0 and the signed rank test is significant.
I would also ask you to read your own answer to OP. How you describe the test makes it clear that it is not a test of the median of the differences.
Sal is correct. Wilcoxon signed rank test compares sums of ranks, not medians. Here's a fictional example where the medians are identical, yet the difference is statistically significant:
Bahar Ysr , You can compare medians, but then you need to use a test that compares medians. Assuming you have paired samples, that might include the sign test, or quantile regression.
The signed rank test is fine, it's just not a test of medians, unless other assumptions are imposed.
- Badges
- Science topic
- Similar topics
- Publishing