Hello.
My research involves analyzing the change between five test intervals on four different composites of a neuropsychological test. I was planning on running four separate repeated-measures ANOVA's. I know that the repeated-measures ANOVA and pairwise comparisons automatically correct for multiple comparisons in one test during analysis, however I was planning on adjusting for Bonferroni's post-hoc to control for Type 1 error from conducting four separate ANOVA's (i.e. the dependent variable used in each ANOVA is part of an 'overall' family of tests).
Unfortunately, my data is not normally distributed and I now plan on running four separate Friedman's ANOVA's and four Wilcoxon Signed Ranks Tests (one Friendman's ANOVA and one Wilcoxon Signed Ranks Test for each composite).
However I'n not sure what to do about Bonferroni's. Do I apply the Bonferroni's correction to control for Type 1 error from running four separate Friedman's ANOVA's post-hoc (i.e. 0.05 / 4)? I know I have to apply Bonferroni's to the Wilcoxon Signed Ranks Tests (the non-parametric equivalent of a paired t-test), for the 10 comparisons from having five test intervals (i.e. Time 1-2; 1-3; 1-4; 1-5; 2-3; 3-4; 4-5; 2-5; 3-5; 2-4). Do I also have to apply Bonferroni's correction to the Wilcoxon Signed Ranks test to control for Type 1 error from running four Wilcoxon Signed Ranks tests, in addition to the Bonferroni's correction for the 10 comparisons within each composite?
Any help would be greatly appreciated!
Thanks in advance!
As Bonferroni correction relates only to the p-values per se, it does not matter if it is parametric or non-parametric clusters of p-values that are corrected. For Kruskal-Wallis can Dunn-Bonferroni be applied.
There is no set way to make corrections for multiple comparisons. Bonferroni corrections is just one way of protecting against the increased risk of type I errors (i.e. incorrectly rejecting the null hypotheses). Whether you do that or not will depend on other factors, such as whether your research is exploratory (Type I errors are more acceptable) or high-stakes such as confirming that a therapy is effective (Type I errors less acceptable), and how much power you have in your research. If you are trying to detect small effects with a small sample, for example, Bonferroni corrections may be inappropriate.
Just do something reasonable. Also, be aware that dropping the significance threshold too much could cause another problem- the increased risk of Type II errors, i.e. incorrectly accepting the null hypotheses. If you decide that correction for multiple comparisons is necessary, you could for just use a 0.01 threshold in place of 0.05 for everything. That is a reasonable strategy to control for multiple comparisons.
No need to worry about normality for Bonferroni. My Fav reference is attached. HTH Good luck, David Booth
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA