I'm interested in studying the effects of two independent variables (Group and Task) on Dependent Variable X. However, due to unforeseen circumstances I've got baseline differences between the Groups that I'm studying (e.g Visual Acuity). Visual Acuity has 5 levels. FYI: I've got 11 and 15 subjects in each of the group.
Just as a form of control, I intend to include Visual Acuity as a fixed effect to examine if there were any possible interactions. However, my biggest problem is that if I include a Group * Task * Visual Acuity interaction in the model, I'm cautious over fishing of data. Firstly, I do not have a specific hypothesis regarding Group * Task * Visual Acuity interaction. Secondly, because of the smaller sample size and number of levels in Visual Acuity - it appears I'm comparing the data of one participant to a group of 3 in one level for example (i.e. it appears pointless to be performing such analysis?)
I've search other websites and some have suggested that it may not be necessary to include every possible interactions and main effects. I was wondering if it make sense to just include these in my model since I'm neither interested in the main effect of Visual Acuity or even Group * Task* Visual Acuity interactions
(1) Group
(2) Task
(3) Group * Task
(4) Group * Visual Acuity
Additionally, I've tested out two models with REML here:
Model 1: (1), (2) and (3), Model 2: (1), (2), (3), (4)
Model 1 with AIC value at 6303:
(1) significant
(2) significant
(3) significant
Model 2 AIC value 6231:
(1) not significant
(2) significant
(3) significant
(4) not significant
If I include the (4) in model 2, what exactly am I actually doing to my data? Should I choose model 1 or 2? if there's anyone that could enlighten me on the above questions, that would be great!!
Dear June Wei,
Very interesting question.
Not everything is black or white... So, yes, you may use model (1) or (2)... I hope I can help you deciding which one you deem reasonable to YOUR question!
Also, I will try not to overwhelm you with things....
To begin with, NO, you are not required to add all possible interactions in the model. Reasons include: your research question, statistical power, representation of levels across variables, interpretation, etc... I won't get push more on this..
Second, I agree with you on 2-way interaction (G*T*VA).. It may be pointless to test that if some of the things I mentioned above make sense to you. With 26 experimental units, 2 G and 5 VA, well, I can see you consuming some extra degrees-of-freedom pretty easily!
Now, the fun discussion.
I think you should have a 5th effect:
(1) Group
(2) Task
(3) Group * Task
(4) Visual Acuity
(5) Group * Visual Acuity
If you fit a model (let's call it "model 3") with (1), (2), (3), (4), and (5), you will have an equivalent model (assuming a few things) as your model 2. HOWEVER, the interpretation of the effects can be different.
In Model 3, we are "adjusting" the data for the differences in Visual Acuity (4), regardless of the Group. What does that mean? Well, regardless of the Group in which the subject is from, the differences in (4) will be accounted for. In other words, the effect of (4) on the response variable X is INDEPENDENT of (1)!!!!! This is, in the vast majority of the cases, a fair assumption.
However, for effect (5), after we adjust for (4), we are asking (one of the possible question, but the most important IMHO): is the effect of Visual Acuity on the response variable X DEPENDENT on the Group???? Well, does the effect of Visual Acuity on X change depending on the Group which the subject is from????????? Well, is this a fair assumption? It could be, but, would you expect that?
Now, maybe more directly answering to your question: In your model 2, your effect (4) is a combination of effects (4) and (5) in my model 3. Therefore, it is not easy to interpret that effect, because you have a combined "Visual Acuity"+"Visual Acuity*Group" effect being tested. Hence, the answer to "If I include the (4) in model 2, what exactly am I actually doing to my data?" is: a bunch of (perhaps, unnecessary) questions based on (perhaps, unnecessary) assumptions.
Therefore, my suggestion to you would be to used model 3, and THEN, you evaluate the significance of effect (5). Ideally, you would reduce this model to model 4:
(1) Group
(2) Task
(3) Group * Task
(4) Visual Acuity
Please note that in this model you are still accounting for potential differences in Visual Acuity that could lead to variation on X. What you NEED to understand is that the fact that there difference in Visual Acuity depending on the Group does NOT MEAN difference in the effects of Visual Acuity on X depending on the Group!! These are two completely different things!!!!!!! If you fit Visual Acuity (and not its interaction with Group) in the model, you are still accounting for the effect of Visual Acuity on X, but you are assuming that these effects DO NOT change based on the Group. See how this has nothing to do with the fact that there is a difference in Visual Acuity between groups?
There are two other important things that I would like to mention:
1) By having missing levels of Visual Acuity between Groups, you make the estimation of these effects and its significance test a bit more complex. This could further support you NOT adding this interaction to your model.
2) Also, perhaps a bit more important, and not necessarily the focus on this question, you shouldn't use AIC under REML to compare models differing in fixed effects, but ML instead. A nice discussion could be found here: https://stats.stackexchange.com/questions/116770/reml-or-ml-to-compare-two-mixed-effects-models-with-differing-fixed-effects-but/116796#116796
Hope this helps! Nick
Dear June Wei,
Very interesting question.
Not everything is black or white... So, yes, you may use model (1) or (2)... I hope I can help you deciding which one you deem reasonable to YOUR question!
Also, I will try not to overwhelm you with things....
To begin with, NO, you are not required to add all possible interactions in the model. Reasons include: your research question, statistical power, representation of levels across variables, interpretation, etc... I won't get push more on this..
Second, I agree with you on 2-way interaction (G*T*VA).. It may be pointless to test that if some of the things I mentioned above make sense to you. With 26 experimental units, 2 G and 5 VA, well, I can see you consuming some extra degrees-of-freedom pretty easily!
Now, the fun discussion.
I think you should have a 5th effect:
(1) Group
(2) Task
(3) Group * Task
(4) Visual Acuity
(5) Group * Visual Acuity
If you fit a model (let's call it "model 3") with (1), (2), (3), (4), and (5), you will have an equivalent model (assuming a few things) as your model 2. HOWEVER, the interpretation of the effects can be different.
In Model 3, we are "adjusting" the data for the differences in Visual Acuity (4), regardless of the Group. What does that mean? Well, regardless of the Group in which the subject is from, the differences in (4) will be accounted for. In other words, the effect of (4) on the response variable X is INDEPENDENT of (1)!!!!! This is, in the vast majority of the cases, a fair assumption.
However, for effect (5), after we adjust for (4), we are asking (one of the possible question, but the most important IMHO): is the effect of Visual Acuity on the response variable X DEPENDENT on the Group???? Well, does the effect of Visual Acuity on X change depending on the Group which the subject is from????????? Well, is this a fair assumption? It could be, but, would you expect that?
Now, maybe more directly answering to your question: In your model 2, your effect (4) is a combination of effects (4) and (5) in my model 3. Therefore, it is not easy to interpret that effect, because you have a combined "Visual Acuity"+"Visual Acuity*Group" effect being tested. Hence, the answer to "If I include the (4) in model 2, what exactly am I actually doing to my data?" is: a bunch of (perhaps, unnecessary) questions based on (perhaps, unnecessary) assumptions.
Therefore, my suggestion to you would be to used model 3, and THEN, you evaluate the significance of effect (5). Ideally, you would reduce this model to model 4:
(1) Group
(2) Task
(3) Group * Task
(4) Visual Acuity
Please note that in this model you are still accounting for potential differences in Visual Acuity that could lead to variation on X. What you NEED to understand is that the fact that there difference in Visual Acuity depending on the Group does NOT MEAN difference in the effects of Visual Acuity on X depending on the Group!! These are two completely different things!!!!!!! If you fit Visual Acuity (and not its interaction with Group) in the model, you are still accounting for the effect of Visual Acuity on X, but you are assuming that these effects DO NOT change based on the Group. See how this has nothing to do with the fact that there is a difference in Visual Acuity between groups?
There are two other important things that I would like to mention:
1) By having missing levels of Visual Acuity between Groups, you make the estimation of these effects and its significance test a bit more complex. This could further support you NOT adding this interaction to your model.
2) Also, perhaps a bit more important, and not necessarily the focus on this question, you shouldn't use AIC under REML to compare models differing in fixed effects, but ML instead. A nice discussion could be found here: https://stats.stackexchange.com/questions/116770/reml-or-ml-to-compare-two-mixed-effects-models-with-differing-fixed-effects-but/116796#116796
Hope this helps! Nick
It depends on your Hypothesis . It is not necessary to include all interactions.
Just we have to verify the model fit statistics if they are proper and model is fit then your hypothesis will decide the number of interactions.whatever interactions you need and significant that can be iterpreted.
- Badges
- Science topic