The pretest scores of a study I'm analysing is significant: the experimental group outperformed the control group in one of the measured dependent variables while the control group did better in another measured dependent variable.
Meanwhile, the experiment (treatment) has been carried out and post test administered.
Since significant difference was observed in both pretest and post test scores of the groups, is there a way of reporting that the treatment produced an effect?
I'm worried the treatment had been concluded and capital is gone.😢😢
Daniel Wright; Burke D. Grandjean, please, do you have a way out? Thanks.
Were the subjects randomized to the groups? If not, the question is impossible to answer.
Is is a repeated-measures design (i.e. each subject is tested pre and post)? If yes, you could simply analyze the differences (post-pre, per subject) to get the treatment effect. If no, the treatment effect would be estimated by the interaction in a two-factorial model (factors "time" [pre, post] and "group" [control,experimental]).
@Jochen, thanks a lot.
No. The subjects were not randomized. Two intact classes were used. The design is quasi-experimental nonequivalent groups design.
Are you saying there is no solution since the subjects are not randomized?
For your first suggestion, could ANOVA serve?
Thanks again.
With quasi-experiments you'd expect the groups would differ on a lot of things. I agree with Jochen Wilhelm suggestion if this is repeated measures. The choice is often between an ANCOVA and an ANOVA on the gain scores. When each is appropriate depends on several assumptions. The literature on Lord's paradox discusses this. In recent years there is much discussion on trying to attribute causality from non-experimental designs. There are several useful textbooks. imho a good intro is Morgan and Winship's book (https://www.cambridge.org/core/books/counterfactuals-and-causal-inference/5CC81E6DF63C5E5A8B88F79D45E1D1B7), but there are several others out there. All these require assumptions and often these are not appropriate. Pearl's Causality is a more theoretical book if you like that. It is often useful to get input from a statistician at your university (ideally this should be done prior to data collection) because there are often many things to discuss.
The advantage of using ANCOVA is that it takes care of initial differences in the groups through the pre-test scores. The stat. is used in a Non Equivalent Control Group Design of Quasi Experimental Study as in your case.
In addition a researcher should not have a pre-determined outcome of his research. Report your findings as they are, so long as your control of the extrenous variables were adequate.
Sylvanus Ugwuda , what do you mean "takes care of"?
I think one of the problems with how ANCOVA is taught is that we use shortened and misleading phrases like "controlled for" when the procedure does not control for like would be done in an RCT. Fifty years ago Meehl wrote about how systematic matching on any set of variables was likely to systematically unmatch on others (http://meehl.umn.edu/sites/g/files/pua1696/f/084nuisancevariables.pdf). It is important not to over-sell ANCOVA. It may be appropriate for some situations but it is not for others (I give an example here: Preprint How Students are Allocated to Schools Should Inform the Mode...
).
@ Prof Wright, thanks for your contributions and the suggested works. Although I have not finished reading them, they're helpful.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS