Hello RG community,
I was having a discussion with a colleague about whether it is better to randomize or not cognitive tests in children. If we randomize, we can calculate group averages of test scores, canceling out fatigue effects, but we cannot compare children within a group for one score as some may have been more tired than others during that test. If we do not randomize, we can compare children scores on a test when they were all equally tired (experiment-wise) but we cannot compare tests as children should always perform better on the test that came first. I want to perform regression analysis between test scores among children and compare children from various backgrounds, therefore I wasn't going to randomize. Then my colleague said "But everybody randomizes tests, you have to !" What do you think ? If you have published examples of not-randomized test sequence in children and arguments for or against randomization, please let me know. Thank you !
Randomization is very important since it is scientifically inclined. So long it eliminates any form of bias that can undermine the credibility of your results, you need to randomize. Though it is risky if you do not have a background knowledge of your children. But in scientific research it is good.
It also depends on whether or not you are using standardized tests. Those that are standardized need to have their sub-tested administered in the prescribed order (e.g., the subtests of the WISC or WJ) WITHIN the test. But across different tests, randomization sounds like a good way to minimize fatigue and practice effects, and to possibly study the effects of fatigue and practice.
Thank you Rhianon Allen for your reply. I am not using tests with subtests, and I am not calculating standardized scores. I am using short tests of executive functions. Studying fatigue effects is certainly interesting but adding this variable to the regression model would require an increase in sample size beyond what I was planning. But why not. Still I am interested in arguments for or against randomization in my situation :)
It also depends on whether you are interested in comparing means or computing correlations. If your main interest is in comparing means, then randomization is useful to prevent bias due to testing order (with the reservations already indicated by Rhianon). However, if you are interested mainly in the correlations among tests, randomizing task order would be an additional source of error variance, which may cause underestimation of the correlations. For this reason, in correlational research, task order is often NOT randomized. Furthermore, it is important to consider the nature of your tasks. If performing task A is likely to affect performance on task B, then it is better to avoid an order in which A precedes B. If your research involves numerous tasks with different content, then it is advisable to devise one (or more) task ordering(s) in which tasks of different types follow one another, to avoid boredom and demotivation (like in the Wechler scales, where verbal and nonverbal subtests are adiministered in alternation).
Thank you very much Sergio Morra . Your argument is similar to mine since I'm performing regression analysis with tasks that are unlikely to affect each other. Yet my colleague says that I should aim for "correlation despite randomization", to be sure. Do you have any published example in mind that would present that kind of design explicitly not-randomized ?
- Badges
- Science topic
- Similar topics
- Medicine
- Epi
- Age Groups
- Children