Jordan R Crawford All the post-hoc tests following the ANOVA contribute to the family-wise error rate - regardless of which groups/levels are being compared.

Blaine Tomkins thank you for your fast and clear reply! Do the post-hoc tests contribute towards (add on to) the family-wise error rate generated by the main and interaction effects? Or is the family-wise error rate separate for the collection of interaction/main effects and those of the post-hoc tests?

Again - probably obvious however if this confirmation is useful to me it must be useful for others with a lack of experience in ANOVA tests who are scouring forums for answers.

Thanks!

Jordan R Crawford The ANOVA does not require an alpha adjustment because the test is designed to make multiple comparisons within a single analysis - including all main effects and interactions. However, it's advisable to adjust the alpha for all the post-hoc tests since each additional post-hoc test increases the probability of a false positive (i.e., false discovery rate) as follows:

P(Type I Error) = 1 - (1 - alpha)n

where n is the number of tests and alpha is the chosen alpha level.

Now, there is some disagreement about whether alpha adjustment is required if, say, two separate tests are intended to assess completely different research hypotheses in a study. However, in this case, all of these post-hoc tests are related to the same underlying hypotheses/questions and the same data, hence the name "family"-wise error rate.