I have two groups (randomly assigned), a control group, and an experimental group. The experimental group received training while the control group didn't receive any training. I collected data for both groups in baseline and retention sessions (pre and post-treatment for the experimental group). I conducted a paired t-test for each group separately to show the significant changes before and after the training. However, the t-test doesn't tell me if the groups belong to the same population. I can conduct an independent t-test between both groups to show that, but my question is: is this the right way to conduct the statistical analysis for this scenario? Will ANOVA be a better choice to show the difference between both groups and the difference within the groups pre and post-training?
Thanks in advance for your help!
Yes, a repeated-measures ANOVA is better than conducting multiple t-tests since you increase the risk of committing a Type-I error with each test. In this case, a repeated-measures ANOVA including 'training' as a between-subjects factor with two levels (training, control) and 'time' as a within-subjects factor with two levels (pre, post). This test will allow you to detect both significant main effects and interactions.
I also recommend conducting Levene's test to determine if the groups have similar error variances. This will give you at least some confidence that the groups are similar.
Hello Hawkar,
I'd like to add a note to Blaine Tomkins 's suggestion. If you use a one-between (group or condition), one-within (pre, post measure) anova, please note that the main effect of group will compare the two batches on the sum of baseline + post scores, and to the degree that baseline scores may have differed, will be a misleading indicator of whether group conditions were equal. In the 1B1W design such as yours, the relevant test is more often that of the group X measure interaction. If present, that indicates that: (a) groups had differing levels of pre-post differences; and/or (b) group differences at baseline were not comparable to group differences at post measure. Only if the group X measure interaction is nonsigificant and groups are equal on the baseline/pre measure, will the main effect of groups accurately evaluate equality of treatment conditions.
An alternate design for the groups comparison would be one-way ancova; such that randomly occurring differences between groups at baseline would be controlled for, and the group/condition comparison would be on post scores adjusted for pre/baseline differences.
Good luck with your work.
Paired t-test will tell you if training is effective or not. You need to compare the data after training with the control group using unpaired t test. If more than two groups you can use ANOVA.
Thank you all for the great answers. I looked into both repeated-measures ANOVA and one-way ANCOVA. I think I understand your answers better after spending some time learning about them. I only have one concern regarding my design, since I only have two observations (pre/baseline and post/retention), will repeated-measures ANOVA still be a good choice? and is Mauchly's Test of Sphericity still be applied? Blaine Tomkins , David Morse
Hello again Hawkar,
For repeated measures factors, Mauchley's test is applicable only if there are more than two measurements/levels of the factor. You have two (pre, post), so it does not apply.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training