Suppose I have four methods A, B, C, and D, and I want to run a longitudinal study to measure participants' performance with each method.
I choose a within-subject design because I have a few number of volunteers. Since I also want to see how performance changes with practice, the study will have ten sessions. In every session participants will execute some task using each of the four methods.
I have read about counter-balancing using a latin square to avoid any interference effect between the methods. However, as far as I understood, this approach seems to be used in cases where you have only one session. For example, consider the 4x4 latin square:
A B C D
B C D A
C D A B
D A B C
In a single-session study, participant 1 would use the methods in the order defined in row 1, participant 2 in row 2, ..., participant 5 in row 1 and so on (the number of participants must be a multiple of 4).
In the example, interference is then assessed by analyzing the effect of "group" (i.e. each order of the methods), a non-significant effect means no interference.
What about the second, third, ... tenth session? Will the order be the same in every session for each participant? Keeping the same order means that participant 1 will always use method D after he or she used methods A, B, and C, therefore performance could be poor with D due to tiredness.
My question is: how to change the order of the methods from one session to the next to avoid interference between conditions?
Can I assume that you are randomly assigning subjects to sessions? If so, it seems that you can randomize the order ABCD for each subject. This randomization approach will ensure that if there is interference, it will be random as well.
Can I assume that you are randomly assigning subjects to sessions? If so, it seems that you can randomize the order ABCD for each subject. This randomization approach will ensure that if there is interference, it will be random as well.
Dear Prof. Ariel, thanks a lot for your answer.
All subjects will have ten sessions, so I think they are not assigned randomly to sessions.
Randomizing the order of ABCD across sessions sounds good. Still, a given sub-sequence could appear more often than another, e.g., for user X, method A preceded method B more often than method B preceded method C.
Following the latin square approach, what if we made "group" a within factor in the study, instead of a between factor?
For example, for user X I would randomize the rows of the latin square and assign rows to sessions.
Thanks and best regards!
Antonio
That would work. There are several variants on the theme, but this is a viable approach. The issue is randomization to eliminate non-random effects.
Thanks again prof. Ariel, for your useful answer!
Best regards
Antonio
A full counterbalance for 10 combinations would require 10! = 3,628,800 combinations--out of the question. Full randomization is unlikely to produce order effects, and few would criticize that approach. That's the easiest way.
If order effects still bother you, then try something like this:
S1: ABCDEFGHIJ S1 and S2 form a pattern-pair
S2: JIHGFEDCBA
S3: ACEGIBDFHJ S3 and S4 form a pattern-pair
S4: JHFDBIGECA
S5: AJBICHDGEF S5 and S6 form a pattern-pair
S6: FEGDHCIBJA
In other words, create pairs of patterns, the first of which runs forward, the second backward. You can either select the first pattern by logical choice, or use random selection-without-replacement.
Either way, you won't get any substantial criticism from any reviewer.
Dear Mr. William,
Thanks a lot for your answer and good ideas!
Best regards,
Antonio
Hi Antonio,
After this exchange a couple months ago, I decided to write a command in Stata to perform this randomization of sequenced treatments. You can download the program from within Stata by typing ssc install randtreatseq
"randtreatseq generates treatments in a random sequence for each individual in the sample, thereby reducing the potential for order effects of multiple treatments."
I hope you find this useful!
Ariel
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Human-Computer Interaction