I am in a bit of a bind with a project evaluating a classroom-based intervention. The design was supposed to be a mixed-design whereby I collected pre-post data from willing students linking through self-generated ID so that I could evaluate whether a particular homework assignment (2 variations) led to different levels of change across time. The problem is that the unique ID did not come through on post-test (because I was the professor and researcher ethically I had to wait to look at the data until after grades were submitted). I am now treating it like a post-test only and looking at whether there are between-group differences based on the assignment completed. However, I am wondering if there is any way to appropriate compare averages from pre-test (either using the average across the sections - there were no differences at pre-test on DV between my 2 sections or appropriate section average at pre-test). I recognize this situation is not ideal as individuals comprising pre-test sample overlap but some may be different than those individuals who participated at pre-test. Maybe one-sample t-test just looking to see if the average changed at post-test from where they started? Any ideas are appreciated.
It seems to me that all is not lost. You can compare the group means on the post-test. While it would be useful to be able to control for baseline scores as a covariate – and this is preferable to using the differences – it's not the end of the world if you can't. The fact that there were not differences in the outcome variable pre-test is reassuring.
I think using a two-sample t-test to compare scores before to scores after would be better than using a one-sample t-test.
The approach I would actually use is to build a linear model like in an analysis of variance. Score is a function of time period and “condition” and the interaction of those terms. If you have other information, like student sex, that could be included as well.
In general, it’s usually better to include all the factors in the model than to run a separate t-test for each hypothesis. In this case, if there is no effect of “condition” or the interaction, the model might just reduce to a t-test for the effect of time period anyway.
Whichever you choose, be sure to check that your data meet the assumptions of the test.
Thank you both for the answers. Salvatore the only challenge I am butting up against is how I would set up my data. Certainly many of the scores at both pre-test and post-test are from the same person. However, I do not have information to match them up. It seems problematic to me to treat them as if they are independent (which is the only way I can envision setting up the SPSS file to allow for an approach as you have described it). However, maybe I am missing something. It seems like beyond looking at pre-test to make sure there are no differences in demographics or variables of interest I might just need to focus on post-test data? What are your thoughts about using a one-sample t-test and using the pre-test average score as a comparison point to look to see if the post-test for the two conditions are different than the pre-test average score?
Hi, Tanya.
No, you're not missing anything. You could use the one-sample t-test. Personally, I would rather treat the pre- and post- as independent samples, even though we know they are not. I'd rather capture the variability in both the pre- and post- measurements, rather than reduce one to just its mean. But I think either approach is reasonable considering the situation.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS