I know I have come across work on how pretests can prime participants as to the nature of the experiment. I have found work on the reactivity of pretests in attitude research, but I would like some literature on the topic in educational/psychological research. Any suggestions for articles?
If you are doing an experiment the control and treatment groups will both react to the pretest. In general, any testing effect is neutralized through the use of a control group. The other option is to use a posttest only design.
Exactly, but could you recommend citations support the notion that students react to the pretest?
Virginia,
You can look at any research design book under threats to internal validity or testing effect. The original source is Campbell and Stanley, 1963.
Hi Virginia,
there are at least three research fields (perhaps some more) where pretest effects are a topic of research, (1) longitudinal studies where tests and questionnaires are administered repeatedly, (2) social psychology, where knowledge (or assumptions) of the experimental goals affect results, and (3) psychometric testing research, where retesting is an issue that affects validity of tests and study results.
You are probably interested in attitude issues, but perhaps I can help you somewhat by recommending you two articles and a book on test-taking effects. I did my diploma thesis (MSci) on a related question. Perhaps you can find methodical papers or research strategies, or citation that link you to you topic when having a look on these texts.
Jensen, A. R. (1980). Bias in mental testing: Free Press New York.
Schmidt, F. L., Ones, D. S., & Hunter, J. E. (1992). Personnel selection. Annual review of psychology, 43(1), 627-670.
Schmit, M. J., & Ryan, A. M. (1992). Test-taking dispositions: A missing link. Journal of Applied Psychology, 77, 629-637.
Hopefully,
could help you out,
Best regards,
Michael Steinborn,
University of Wuerzburg
www.michaelsteinborn.de
Hi, Virginia,
one useful way to reduce the effect of pretests is to employ measurements with a low face-validity. If people believe to respond to something else than what you are actually investigating, you may be more confident that responses will not be biased. Nicier that that is to combine both measurements, one with high and another low face validity in pre- and post-tests. The discrepancy between both tasks will give you a measure of the bias. With that in your hand you can use the pre-test values as a covariate when evaluating differences between experimental and control group in the post-test. If your n is not too small, this would be sufficient, if not to prevent bias, at least to characterize it and partial it out before comparing the genuine effect of your intervention.
Great answer Michael Scott. I always suspected there was a better research design, but had never learned it or come across it until your answer.
You may also want to consider doing a factor analysis of the instrument on a pilot group. This will give you some insights into whether you should change items.
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning