I plan on running an online perceptual experiment that has, let's say, 100 unique video stimuli altogether. Since these are too many to be rated by every participants, I would like to present the participant with 50 stimuli from that pool and therefore create several predefined sets (so as an example, Set 1: Stimuli 1, 2, 3, 4; Set 2: Stimuli 3, 4, 5, 6; Set 3: Stimuli 1, 2, 5, 6 -- Participants 1-5 get Set 1, Participants 6-10 get Set 2, Participants 11-15 get Set 3). Participants are asked to rate several perceptual qualities on 7-point scales.
I now wonder which statistical method(s) would be appropriate and possible to use with such a stimulus configuration, as I guess regular ANOVA or repeated measures ANOVA are out of question due to the participants not rating every stimulus available. Would maybe linear mixed models be a way to go, or other methods? What would be other issues to keep in mind (e.g., large sample size)?
Thanks for any hints, previous threads, articles with the same issue, etc.!
Dear Birgitta Burger,
Repeated measures ANOVAs and MANOVAs do not require that each participant answer every questions. A repeated measure ANOVA is used when each participant provides several evaluations of one single dependent variable. Based on your description I believe a Multivariate ANOVA (MAVOVA) could be better appropriate. It is used when there are multiple dependent variables (in your case it could be the perceptual qualities).
I would need more information regarding the aim of this experiment to provide you with a more specific answer. For instance, can the videos be classified in sub-groups or do you want to compare the 100 videos to one another? Do you want to compare the different perceptual qualities?
Hi Birgitta
If the full stimulus has a systematic variation, you should incorporate that into the sub-sampling (e.g., fractional factorial designs). If you fully have random variation between the subsets, you should be able to analyse the full set with partial data from the subsets using mixed models as long as the participants have been exposed to the full range of the variance in the data in their subset. I can’t immediately recommend an article/chapter to flesh out my argument but if you look at multi-level GLMM models, the problem is often similar.
- Badges
- Science topic
- Similar topics
- Psychology
- Cognitive Science
- Mind