Hi,
I have a question about statistical analysis, specifically ANOVA and the paired sample t-test.
I’m currently preparing an experiment, in which I will measure RTs in a task that involves pictures and sentences. The participants will read a sentence and then see a picture. Their task will be to decide whether the object in the picture was mentioned in the sentence. There will be four picture-sentence combinations and I will create four lists so that each group will see only one of the possible combinations. Hence, this will be a 2 (sentence type 1 or 2) x 2 (picture type match or mismatch) x 4 (lists) design. In this setup, picture type and sentence type are the within-subjects variables and list is a between-subject variable.
Ostensibly, this is a multi-factorial design, but effectively I’m only interested in the match x mismatch interaction (i.e. whether participants are faster in matching conditions). List is a dummy variable and I won’t be analyzing how sentence type or picture type affects the variance of the data. I only want to compare two means: mean RT in matching conditions and mean RT in the mismatching conditions.
Can this be done simply with a paired sample t-test or do I still need to run a mixed design ANOVA? If both are possible, would there be any advantage of doing a full ANOVA over a paired sample t-test? Most of the similar research that I’ve read up on uses mixed design ANOVA and I’m curious if there’s a good reason do to that.
I do realize that running multiple t-tests significantly increases the probability of type I error, but is not clear to me whether the same thing happens in this type of design, since only two means are effectively compared.
I’d really appreciate if anyone could clear this up for me.
As I understood it, "lists" should be a random factor, and then in fact all you are interested in is the interaction in the fixed 2x2 part. That's one model and one test (there is no need to consider whatever else a software will spit out for that model).
It seems your response variable is binary. Instead of ANOVA you should then better use a generalized linear model (GLM) of the binomial family, that is, a kind of a logistic regression model.
As I understand, you have 2x2, 'doubly' repeated measures (within-subject), i.e. each subject will receive 4 sentence-picture combinations (i guess, in randomized sequences; e.g., for 1 subj.: 1-3-4-2), for a 2nd subj.: 2-4-3-1, etc.):
1) sentence type1 - picture matched
2) sentence type1 - picture mismatched
3) sentence type2 - picture matched
4) sentence type2 - picture mismatched
You further have 4 lists corresponding to a between-subject or group variable.
I would run the mixed design (2x2 within, 4 between) ANOVA. You expect only 1 main effect: picture type (matched vs. mismatched) and no effect of list type, nor of sentence type, nor any interaction (pictureXsentence, pictureXlist, sentenceXlist, pictureXsentenceXlist). I guess you expect the RT to be sign. faster for matched type than for mismatched type. The advantage of running the mixed design is that it replicates 'similar research' and that, if you only have 1 sign. main effect (that of picture type,) your conclusions are better founded. You take the risk of finding another sign. main effect or a 2-way or 3-way interaction, which would be puzzling or complicating.
First, you referred to match versus mis-match as an interaction effect, but is a main effect. Second, if you are including sentence type because you hypothesize that it has no effect, then you need to test both whether that main effect exists and whether it interacts with match versus mis-match. Beyond that, I agree with Jos Feys about how to handle the lists.
Your response variables are response time and response accuracy, so you also need to consider how to model these together. Since it appears it is only a binary choice there are lots of models created by cognitive scientists for situations like this. Probably the most developed of these is Roger Ratcliff's diffusion model. If you google that you'll see lots. There are other approaches though (and if almost all responses were correct it also affects analyses). The next step will be building in the within-subject factor design as mentioned by the previous commentators. On this aspect, was it the same stimuli used for each person?
- Badges
- Science topic