If the interaction effect is significant, you can do a post-hoc test that compares all four treatment combinations. If your software has an lsmeans or an emmeans option, this is a good approach in general. If not, Tukey's test on the interaction is a good option.

In my opinion, there is no need to do a post hoc test for just two groups. You can describe group-specific differences using the descriptive statistics output associated with the results.

The significance of the interaction allows you to reject the null hypothesis that the two simple effects are equivalent. Sometimes, people might follow up with contrasts to ask if each simple effect is significant or not. As you are in psychology, it may be that you use SPSS. If so, you can get tests on the two simple effects via the EMMEANS sub-command for UNIANOVA. Suppose your two IVs are A and B, with A as the focal variable and B as the moderator (or effect modifier). The code would look something like this:

UNIANOVA Y BY A B /EMMEANS = TABLES(A*B) COMPARE(A) /DESIGN = A B A*B.

That COMPARE(A) will give tests comparing the 2 levels of A at each level of B.

HTH.

Thank you all for the quick responses! This is very helpful!

Sal Mangiafico does this also mean that I in general do not follow up with a post hoc test for main effects, only for interaction effects? In this case, an independent samples t test would not be the right follow up for a significant main effect? Regarding the Turkeys test, I tried it in R and SPSS but always get error responses that I have too few levels in my IVs (2 levels each). But I will look into the emmeans function.

Bruce Weaver thank you for the code! I mainly use R but am also familiar with SPSS. I looked into the emmeans function and just have one more question for clarification. In the emmeans function, do I choose to display only the means for the interaction (referring to tables A*B) or also for each IV if I have a main effect as well? What exactly do you mean regarding "COMPARE(A) /DESIGN = A B A*B"?

Hello Nina Sue. I bet Sal Mangiafico (or another one of the useRs) can show you how to do the same thing with R. But to answer your question, you can add more EMMEANS lines if you want to show the main effects for A and B.

UNIANOVA Y BY A B

/EMMEANS = TABLES(A)

/EMMEANS = TABLES(B)

/EMMEANS = TABLES(A*B) COMPARE(A)

/DESIGN = A B A*B.

You could add COMPARE to each of those lines too, but in a 2x2 design like yours, the tests will be equivalent to the F-tests you already have for the main effects.

• Yes, the general advice is that if the interaction is significant, you don't need to conduct post-hoc tests on the main effects. Simply, it's because post-hoc tests on the main effects won't be of much interest since you know that there is an interaction effect. But there's no law about this.
• E.M. means in R are easy to get. For example,

library(emmeans)

marginal = emmeans(model, ~ Instructor + Supplement)

pairs(marginal, adjust="tukey")

• In R, it appears the the TukeyHSD function in the native stats package works fine when there are only two levels of a treatment.
• The results from emmeans with the "tukey" adjustment will be the same as the classic Tukey HSD test for a standard two-way anova.
• In general, I would avoid using pairwise tests (e.g. pairwise t-tests) for post-hoc testing. It's better to use post-hoc approaches that take into account all the data together. But also, when you have very flexible approaches like using emmeans, there's no need to worry about Tukey's HSD or Dunnetts, and so on.

Thank you Bruce Weaver and Sal Mangiafico for your explanations! I highly appreciate it. This was really helpful.