I have conducted an educational intervention study using a quasi-experimental control group design. I have conducted data collection before and 2 weeks after the intervention. Before conducting analysis, I assessed homogeneity of the outcome variable. There is a statistically significant difference between the mean score of the control group and intervention group in the pre-test independent t-test analysis. In addition, there is non-normal distribution of outcome variable during the post-test. In this circumstances, how the pretest outcome variable can be controlled to assess the effectiveness of the intervention?
The control you want is best given by a standard pre-post experimental design.. this is often difficult in a classroom situation. That's why you chose the quasi approach.. this gains classroom ease and costs some of the control you desire. One hopes that this is going to be close enough. Sometimes not. That's why we use standard. designs to be sure that we get that control
BTW. The difference scores should be aproximately normal not the individual scores. Best wishes, David Booth. See the attached screenshot search for more details.
Hello Prativa,
Alas, no design is perfect, but when one can't implement randomized designs, more "threats" to study validity are an unavoidable part of the landscape (as Professor Booth's reply implies).
In the absence of other information, the best you can do is to use pre-treatment scores as a covariate, and compare groups on adjusted post-treatment scores via ancova (or the regression analog). If distributional assumptions are not satisfied, you can use bootstrap/resampling approaches for a robust comparison without distributional concerns.
Good luck with your work.
The choice of whether using the pre-treatment as a covariate, subtracting it, or something else in your analyses depends on other assumptions (I am not sure if @David Morse is saying that one is always the best, I assume not since that would be wrong and is much discussed in this literature, particularly in psychology, e.g., Wainer 1991, Psych Bull). The one factor that I have looked most at is what factors influence the group allocation (and how they relate with the pre-treatment). I discuss this in a few places in relation to educational intervention, most recently in degruyter.com/document/doi/10.1515/edu-2020-0107/html , so the references are pretty up-to-date. I recommend in particular Rubin and Holland, Pearl, Wainer, and Lord (oh, and and Meehl). Rubin makes that point in this and in general in the Rubin-Holland-Neyman framework for causality that decisions often rest on untestable assumptions.
ps. Also lots of textbooks on making causal statements from quasi-experiments. Some of these are cited in the above also.
pps. Also, since this is an education intervention I assume students were nested in schools or classrooms so make sure you take that into account with mixed models or something.
Just to keep the discussion going optimal experimental designs do exist. See the attachment for helpful details. Best wishes, David Booth
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size