I am doing research to compare learning model of A, B, and C on students mathematical ability improvemnet. However it turns out that the pretest result shows significant differences among class that use learning model A, B, and C (score A> score B; score A > score C; and Score B = score C)
Meanwhile posttest also shows significant different that score A = Score B; Score A > score C; and score B > score C.
Is it okay to still compare the gain score of the groups using Anova?
Or shall I use pretest as covariate using Ancova?
You could indeed run an ANCOVA, which in theory 'corrects' for the differences in pretest. Yet, an essential assumption for the ancova is the 'homogeneity of slopes'. See
https://statisticsbyjim.com/anova/ancova/
This assumptions staes that the correlation between pre and post should be about equal between groups.
Hi Muhamad Syahidul Qirom. If I follow, different classes were allocated to 3 different teaching methods (A, B, and C), right? How many classes were allocated to each method? And was it random allocation? Thanks for clarifying.
Hi Bruce Weaver
One class for each method. it was not random because my research design is nonequivalent control group.
Thanks for clarifying, Muhamad Syahidul Qirom. I was wondering if you could use a multilevel modeling approach, but with one class per method, I don't think so. ;-)
If you have not done so already, I suggest making a scatter-plot as follows:
- Let X = baseline score, Y = follow-up score
- Use distinct marker shapes or colours for the 3 groups
- Plot regression lines by group, letting each group have a different slope
- Add a vertical reference line on the X-axis at the grand mean of the baseline scores (i.e., the mean for all scores, ignoring group).
Then upload that plot here so interested readers can have a look at it. The main things I'll look for are these:
1) Are the lines for the 3 groups close enough to parallel that I would be comfortable with a model that forces them to be parallel (i.e., the classical ANCOVA model)? Or would it be better to include the Group * Covariate product term in the model so that each group can have its own slope?
2) If you are comfortable with the ANCOVA model (which forces the slopes to be equal), another question is whether it is sensible and defensible to compare the adjusted group means where X = the grand mean of the baseline scores. Given that one group has higher scores than the other two, that vertical reference line at the grand mean of X might be passing through some very sparsely populated territory for one of more of the groups. If so, you may want to reconsider your approach.
I hope this helps move things along for you.
The ANCOVA v Gain Score decision is often called Lord's Paradox, and there is a lot on it. It depends on a few things, including what (unmeasured) variables affect the group allocation (since it is non-random) and the other variables. I describe some of the issues here: Article Gain Scores, ANCOVA, and Propensity Matching Procedures for ...
. As @Bruce says, scatterplots are the first step.- Badges
- Science topic