I can't really wrap my head around this and maybe somebody can help me.
I conducted an experimental study with a 2x2 between-subjects design (n = 50 per condition). I have the following variables:
- Fixed factor 1 (dichotomous, experimental manipulation of stimulus)
- Fixed factor 2 (dichotomous, experimental manipulation of stimulus)
- DV (metric, a scale from the questionnaire)
- Covariate (metric, a scale from the questionnaire)
- Moderator (metric, a scale from the questionnaire)
I first ran the base ANOVA model with the following terms:
- Factor 1
- Factor 2
- Factor 1 * Factor 2
- Covariate
Nothing was significant. Out of curiosity, I included the moderator by adding the following terms in addition to the above:
- Moderator
- Moderator * Factor 1
- Moderator * Factor 2
- Moderator * Factor 1 * Factor 2
What happened is that Factor 1 * Factor 2 became significant (after being non-significant in the base model) as well as the three-way interaction Moderator * Factor 1 * Factor 2.
So the 2 issues I am struggling with are the following:
For the two-way interaction, I guess that I would report the marginal means of the DV in the 4 conditions and plot them with lines. This option is provided by SPSS.
But for the three-way interaction, if I understand correctly, the above two-way interaction between experimental conditions is not uniform, but differs across various levels of the metric moderator (e.g., the two-way interaction might be reinforced/attenuated/reversed/non-significant at certain values of the moderator). But SPSS doesn't seem to offer an option to plot this in the ANOVA menu.
I know that the PROCESS macro by Hayes features conditional effects at -1 SD, mean, and +1 SD values of the moderator as well as Johnson-Neyman areas of significance. So I guess I would need something in this direction. Or am I mixing up things here?
I assume you are using the UNIANOVA (or equivalent GLM) command. If so, you can use multiple EMMEANS sub-commands to get tests of the 2x2 interaction at selected values of the quantitative variable. Here is a rough approximation of what the code would be.
UNIANOVA Y BY A B WITH M /EMMEANS = TABLES(A*B) WITH(M=value1) COMPARE(A) /EMMEANS = TABLES(A*B) WITH(M=value2) COMPARE(A)
/EMMEANS = TABLES(A*B) WITH(M=value3) COMPARE(A)
/DESIGN A B M A*B A*M B*M A*B*M.
Replace value1, value2, and value3 with the values of M you want to plug in. And if you want to plug in more than 3 values, just add some more EMMEANS sub-commands.
I've assumed that you want to look at the simple main effect of A at each level of B with M = a given value. If you prefer to look at the simple main effects of B at each level of A, change COMPARE(A) to COMPARE(B). I've also assumed that you want to follow the usual practice of including all lower order terms involving A, B, and M, the terms involved in the highest order interaction.
Good luck. I hope this helps.
Not only in regression, but also in ANOVA, it is crucial to center your continuous covariates (as in ‘A by B with covariate’). That is, when you also request interactions between such a covariate and a main effect in the ‘/design’-subcommand. Otherwise, it may indeed occur that a lower term interaction may become false significant through adding a higher term interaction, as you described. In fact, without centering, your results are nearly uninterpretable. The main purpose of centering is to make the main-effect terms and the interaction-effect terms independent of each other, so they can be evaluated separately. This is also crucial in ANOVA with covariates, and this has received far too little attention. Not centering will give you counterintuitive results, as you experienced already.
To see what a three-way interaction might look like in practice, I made an exaggerated example in Figure ‘Three-way interaction example.png’. To illustrate, the DV increases from A=1 to A=2 in the B=2 subgroup when the Moderator=1, but decreases in the B=2 subgroup when the Moderator=3. (The Moderator here is ‘continuous’, be it only with values 1, 2, and 3 for illustrative purposes). In other words, the interaction between A and B depends on a third factor, in this case the ‘Moderator’.
In ‘Annotated SPSS-syntax with three-way interaction.docx’ you may see how you can make a Figure at the values of Moderator = M-1SD and M+1SD (there is no need to study and use Hayes’ PROCESS to do this). The result is (again) that the DV in subgroup B2 increases from A=1 to A=2 when the Moderator is ‘low’ but decreases when the Moderator is ‘high’. The A*B interaction depends on the value of the Moderator (see: ‘Three-way interaction example with Moderator at M-1SD and M+1SD.png’).
- Badges
- Science topic