I have two measurements of a variable per observation, one pre- and one post-treatment. Although my IV (Treatment) has more than 2 levels, my question does also apply to te case with only 2 levels.
I can test for pre- post-differences (within-subject) with a signed rank test.
To test for between-group differences, I want to take pre-treatment measurements into account.
Is using a MWU on change scores a valid approach?
I know that there are alternatives, but would like to focus on the pro's and con's of the MWU on change scores.
Thanks in advance.
The test test the hypothesis that Pr(Delta>0)=0.5 (i,.e. "stochastic equivalence of Delta and 0"). So the answer is: yes, if this is the hypothesis you want to test.
I think you can use something like Mann-Whitney for difference in scores.
However, I can think of some reasons why you may not want to. For one thing, by using the difference in scores, you are losing information on the absolute scores.
Consider the plot linked below. Ignore the error bars. If you look at difference in scores across treatments, you would probably find that the green and the red have similar difference in scores. But you have lost the fact that green has low scores before and after and red has high scores before and after. Red at Time 1 may be significantly higher than green at Time 1.
You can put all the data into one appropriate model to look at things this way. The dependent variable is Score.
But if you are interested only in the difference in scores, you could use the Difference in Scores as the dependent variable, and use an appropriate test to test among treatments.
http://rcompanion.org/handbook/images/image088.png
P.S.
I don't really like the idea of using Scores as the dependent variable for a paired rank sum test, and then using Difference for the dependent variable for a M-W test.
An alternative is to present the Difference in Scores across treatments, with an appropriate test. And include confidence intervals for the medians (or means) for Difference for each treatment. A confidence interval including zero suggests the change for that treatment was not significant.
This seems cleaner to me, and more analogous to how we present regression results.
Be sure to use appropriate methods for confidence intervals. If your data is not normal, you probably want to use medians and use bootstrapped confidence intervals.
MWU, or Kruskall-Wallis for more than 2 treatments, just compares the differences between the groups, without allowing for any other covariate. So if you want to use the starting level as a covariate (« I want to take pre-treatment measurements into account »), you cannot use it.
I am not aware of a rank-based test, like MWU, that can take into account continuous covariates. However, you can use a classical linear model approach, but compute confidence intervals or p-values or whatever you want using bootstrap or similar distribution free methods. You may have to be very careful about what you're testing in such an approach, and having a sufficient sample size.
You may also use random effects ordinal models to use scores directly as ordinal variables.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics