There were five pretests administered to both groups. Each pretest represents a different topic in chemistry (Matter and Its Interaction). In lesson 1, the experimental group has significantly higher group means than the controlled group. In lesson 2, two groups have statistically equal group means. In lesson 3 and 4, the control group has significantly higher group means than the experimental group. In lesson 5, the two groups have statistically equal group means. In the overall pre-summative test, the two groups have statistically equal group means.
Welcome to the messy universe of educational research! Your problem, you will be glad to know, is a common one1 : interventions have to be delivered to pre-existing blocks of students such as classes and it's quite likely that these will differ on the outcome variable even before the intervention.
Furthermore, the passing of time will also affect scores. So comparing post-study means in the intervention and control groups is made uninterpretable by the pre-existing difference and by the possibility that the passing of time had a different effect.
The short answer here is that you can approach the problem by looking at the difference in the differences between the groups. Luckily, Stata has an excellent coverage of difference-in-difference models and even if you don't use Stata you can download and read their excellent manual on the method, which includes both nontechnical description and worked examples, as well as, of course, methods and formulas and references.
https://www.stata.com/manuals/tedidintro.pdf
1 That doesn't meant there's a simple solution, but it means we all share your pain.
Hello Mohamed Khedawy. Using a statistical test of normality as a precursor to a model that assumes normally distributed errors1 is not a very sensible thing to do, IMO. Normality of the errors is most important when n is small, and becomes less and and less important as n increases. And tests of normality, like other tests, have low power when n is low and increasing power as n increases. Therefore, your test of normality has low power to detect non-normality when non-normality is most important, and too much power to detect it when it is not very important. This is the main point of a short conference presentation I gave on this topic some years ago:
Presentation Silly or Pointless Things People Do When Analyzing Data: 1. ...
1 Notice that it is the errors (not the Y scores) that are assumed to follow a normal distribution. Note too that nothing in the real world is truly normally distributed. The best one can hope for is approximated normality. See George Box's classic paper, Science & Statistics. Fortunately, linear models are quite robust to non-normality of the errors. Independence (and homoscedasticity) of the errors is far more important than normality of the errors, particularly as n increases.
Box, G. E. (1976). Science and statistics. Journal of the American Statistical Association, 71(356), 791-799.
https://www-sop.inria.fr/members/Ian.Jermyn/philosophy/writings/Boxonmaths.pdf
- Badges
- Science topic
- Similar topics
- Chemistry