Hello folks!
We created two training courses: A (N=25) and B (N=25). Participants were assigned to one of the two groups respectively. They were tested before, shortly after, and 3 months after the training: T1, T2, T3. A score was calculated per participant and test, which ranges from 0 to 62 (ratio scale). The data has the following structure:
User | Group | T1 | T2 | T3 | Score
1 A 25 40 37
2 B 25 41 36
...
The research question is: Which approach helped participants perform better? In other terms: Do groups change differently over time?
So, in the end, I want to form paired differences between tests and compare them group-wise to answer which group performed better in which phase, e.g., group A performed sig. better in the short-term (T1-T2), there are no significant differences in the mid-term (T2-T3), B performed sig. better in the long-term (T1-T3).
I saw this article which has basically the same target: https://statistics.laerd.com/spss-tutorials/mixed-anova-using-spss-statistics.php
So, I used a mixed ANOVA; it considers the within-subjects factor Test which has 3 levels (T1, T2, and T3), and the between-subjects factor Group which has 2 levels (A and B). As can be seen on the screenshot, Mauchly's Test is significant, so I corrected the degrees of freedom with Greenhouse-Geisser estimates of sphericity.
For "Test * Group" I retrieve F(1.472, 70.663) = .644, p = .482, which tells me that differences between Group and Test were not significant. So the two groups did not change differently over time.
Are both, the procedure and the conclusion, correct?
Thank you in advance!
If all other assumptions are met, I would say your conclusion is correct. I would additionally look at the relevant pairwise comparisons, not to test them but to assess the effect sizes and report them.
Thank you! Is it common to report effect sizes when there is no statistically significant difference?
Hello Ionnis,
Your interpretation is correct for the analysis as implemented. About the only thing I might have thought about doing differently is having used baseline (T1) scores as a covariate, such that the repeated measures dimension would have two, and not three, levels (T2, T3) associated with it.
And, yes, ES may be reported (as Rainer suggests), though with the appropriate CI estimate for it, you'd see that the value crosses zero, no matter what the point estimate turned out to be.
Good luck with your work.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA