Hello everyone,
I have conducted a research about applying some teaching method, I had two groups of student, the first one has been taught using a traditional approach, while the second has been taught using a new approach. Both groups undergone a pres-test, then a post-test has taken place. The samples have been randomly chosen; and the tests have been anonymous, so there is no matching between the pre and post marks of students.
On the whole, I have 25 students in each group, please see attached data file.
Could anybody, please, tell me about the most appropriate statistical test to be applied? I am a bit confused about applying either T-test or Z-test.
Thank you in advance.
There is a "post-pre effect" in the experimental group, and there is also some "post-pre effect" in the control group. What you are interested in is the difference of these two effects (experimental vs. control). This is obtained as the interaction in a two-factorial linear model (Time with "post" and "pre" is on factor, and Group with "control" and "experimental" is the other factor; the Time:Group interaction is what you want.
In your data, the coefficient estimate for the interaction is 3.98 with a standard error of 2.9 (on 96 d.f.). The t-value is 1.36 and the p-value is 0.177. The residuals of this model look fine, they are approximately normal distributed without outliers.
Conclusion: your data is not sufficient to interpret the sign of the coefficient estimate: we cannot say if the treatment increases or decreases the values.
I did the calculations in Excel and attached the file (fitting such models in Excel is a pain; one has to manually create the design matrix and take care of the inverted order of coefficients returned by LINEST). The red values are the interesting ones (for your research question).
@ Jochen Wilhelm :
Thanks a lot for the well-explained answer; but I still have a simple question:
Can we apply a t-test between the data of the experimental group before and after getting the treatment? Or need we consider the whole set of data samples? The same question for the comparison between the data coming from the control and experimental groups after having gotr the treatment.
Thank you very much, especially for provided excel data.
@Amer:
although you surely can do so - it woudn't help you answering your scientific question. Consider following cases that can happen when you do individuals tests ranther than testing the interaction:
- testing expt group after vs before: even if this is significant, you don't know if the time has some effect. To control for this, you did the control group. If you' test also the ctrl group after vs. before, and this was also significant, you would still not know if your data allows you to conclude the sign of the treatment effect. And if is was not significant, your data is not conclusive regarding the change over time alone.
- testing expt vs. ctrl at the "post" time point ignores that the values at the "pre" time-point could already have been different. So you don't know if the difference you see can be attributed to the treatment or to differences that existed between the expt group and the control group already prior to the experiment.
@ Jochen Wilhelm :
Thank you very much, everything is clear now. It only remains that I need to understand the notions related to the factorial modal, to know how to apply this type of tests on future data. Thus, I would be really thankful if you could provide me with some sources describing this model and when to apply it.
Thank you in adavance.
@Amer
Are your data for pre- and post- tests ordered the same for individuals? Is number one in pre- the same individual numbered one in post- and number two in pre- the same individual numbered two in post-, etc?
Individual changes are often more important than group changes.
@Amer,
That's basic statistics, you will find it in any (good) stats book, and there are thousands of resources in the internet. Google for "linear models" or "general regression models". This could be a start:
https://onlinecourses.science.psu.edu/stat501/node/2
Order your data in each of the four data sets in ascending order. Make the assumption that the rank of individuals is the same in pre- and post-tests. Compare quantile plots of the sets. There is a different data structure between controls and experimental. The structural change from pre- to post- is different.
Do the differences appear to be important? Do you learn anything from the differences?
It is important to track individuals in order to observe what changes in the groups occurred.
Matched pair/ dependent sample t-test is appropriate for pre-test and post- test data of the same group.
I have no idea, mainly I studied qualitative research, action research and teacher researcher in the qualitative and case study, it is true some of my publications include statistics but I got external help in doing them, thanks
Check this link may be helpful
http://www.jerrydallal.com/lhsp/prepost.htm
regards
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis