A Phd student of mine wants to see whether an innovative English language method he applies has any impact on students' learning. He is checking this using t-tests with the various competence assessments he asked students to complete before and after his intervention. However, one of the groups he taught was English absolute beginners - so in this case, he didn't perform any pre-intervention test; he only has the result of the post-intervention assessments. Can he still perform a t-test? If yes, shall he indicate "0" as the pre-test scores?
Hello Chrysi,
Presuming the absolute beginner group to have uniformly scored "0" on whatever pre-test might have been used is likely inaccurate. The question is, whether that inaccuracy is slight, moderate, or serious. So, my recommendation would be to use that batch's post-training results in a descriptive manner only (no inferential test).
The ideal approach for such a group would have been to randomly assign them to subgroups. Then, expose one subgroup to the target method, and the other subgroup to the standard method, and compare both on post scores.
Using the other (non-beginner) batches in a pre-post analysis should be fine, but unless one has a baseline indicator of the efficacy of the standard instructional method, all your student can say is whether post scores were different from pre scores, with the qualification that such differences are presumably due to the instruction given, and not something else (e.g., cooperative study outside of class, independent reading or practice, etc.).
Good luck shepherding your student to completion.
Professor Morse's advice is on the mark; NEVER use the typical one sample t-test value of 0 unless something is starting at absolutely 0 (even by guessing, participants would usually score something). There is one possibility that could work. If these competence assessments have been tested for validity and reliability (usually done by the instrument's developer) there may be a pre-test average score already established. If so, that value would be entered as the test value to check for variation of your student's subjects. That would be ideal to see if the intervention is statistically significant. Reminder: have student include this information in their methodology section of the final dissertation manuscript.
Good luck with all of your dissertation advisements. I was on over 175 committees (Advisor /Mentor/Chair for 43) during my career and everyone of them were personally and professionally rewarding; especially to get the candidates to completion and graduation. 😉 👩🎓👨🎓