I recalled that it's all right to maintain the alpha at .05 without Bonferroni correction. Is Bonferroni adjustment absolutely mandatory to examine the simple main effects. For instance, for a 2 (sex) x 2 (race) fully independent design with significant interaction.
Hello Jimmy,
For a 2 x 2 design, there would ordinarily be no need for post hoc tests unless you had evidence of a significant interaction. The reason is that each main effect has only two levels, and a post hoc test (for a main effect) would be completely redundant with the test of the main effect itself.
Presence of an interaction means that you might well be interested in evaluating the 4 possible simple effects tests (Factor A levels compared only at level 1 of factor B, then again only at level 2 of factor B; then reverse to compare levels of factor B at level 1 of factor A, then again only at level 2 of factor A).
So, how much risk are you willing to live with of a spurious rejection of a null hypothesis? Four (more) tests, each at the .05 level, imply that the aggregate risk of type I error (for the set of four tests) could be as high as about 18.5%. That's the reason why people choose to invoke adjustments when running additional tests on the same data. How much of an adjustment (and Bonferroni, while simple to apply, is probably too conservative in many cases) is a matter of taste. If you chose the .025 level for each individual comparison, then aggregate risk for 4 tests would not exceed 9.6%. It's up to you.
Good luck with your work!
Hello Jimmy,
For a 2 x 2 design, there would ordinarily be no need for post hoc tests unless you had evidence of a significant interaction. The reason is that each main effect has only two levels, and a post hoc test (for a main effect) would be completely redundant with the test of the main effect itself.
Presence of an interaction means that you might well be interested in evaluating the 4 possible simple effects tests (Factor A levels compared only at level 1 of factor B, then again only at level 2 of factor B; then reverse to compare levels of factor B at level 1 of factor A, then again only at level 2 of factor A).
So, how much risk are you willing to live with of a spurious rejection of a null hypothesis? Four (more) tests, each at the .05 level, imply that the aggregate risk of type I error (for the set of four tests) could be as high as about 18.5%. That's the reason why people choose to invoke adjustments when running additional tests on the same data. How much of an adjustment (and Bonferroni, while simple to apply, is probably too conservative in many cases) is a matter of taste. If you chose the .025 level for each individual comparison, then aggregate risk for 4 tests would not exceed 9.6%. It's up to you.
Good luck with your work!
Hello Jimmy
1. The above answer of David is sufficient and completed.
2. May I can add the following remarks:
i. Two-way analysis of variance allows you to compare the means of a dependent variable when there are two independent variables.
ii.If you have more than one dependent variable then you simply repeat the analysis for each dependent variable separately. On the other hand, if the several dependent variables are measuring much the same thing then they could be combined into a single overall measure using the summing procedures described in Chapter 36 or use MANOVA.
iii.The technique is used when you have more than two means. It adds no further information if there are only two means.
iv. ANOVAs with three or more means to compare are almost certain to benefit from employing multiple comparison methods.
v.It is not possible to give definitive advice as to which multiple comparison test to use in different situations.
3. For more information please refer to:
i. Cochran,W.G. (1977). Sampling techniques, Wiley.
ii. Snedecor, W.G. and Cochran,W.G. (1989). Statistical Methods, Blackwell.
iii. Landau, S. and Everitt, B. S.(2004). A Handbook of Statistical analyses using SPSS.
.Howitt, D. and Cramer, D.(2008). Introduction to SPSS.
Regards,
Zuhair
Hi Jimmy. I mostly agree with David here. I will add that some post-hoc tests such as the Tukey's HSD test will inherently correct for family-wise error rate (i.e. multiple pairwise comparisons). The assumptions for using the Tukey's HSD test are similar to those for ANOVA, so it should be appropriate for your case.
https://stats.stackexchange.com/questions/33357/does-the-tukey-hsd-test-correct-for-multiple-comparisons
All the best,
Jordan
The 2x2 ANOVA has three degrees of freedom. If these 3 df are also from the same "family" then, like the 4 group ONEWAY ANOVA you might choose to adjust the p-values (or the alpha) to protect yourself from any Type 1 error (assuming just looking at all the coefs, not the overall model fit or a single one, like the interaction). Here is some code showing this for 10,000 replications. I'll use R since it is freely available. Also, I'll use a for loop since more people are used to reading these (there will be faster ways to code this).
> rep n issig issigholm set.seed(1291)
> for (i in 1:rep){
+ x1
Two other bits. I don't know where your p = .25 comes from. Can you explain this. And I would not recommend Bonferroni's (see @David's point about it being conservative). For the situations where Bonferroni's is applicable, Holm's is more often now used as it also controls for Type 1 error, but is more powerful.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA