Hello there!
I'm running multiple 2x2 Mixed ANOVAs (2 groups, 2 time points). I have 10 dependent variables so I ran 10 2x2 Mixed ANOVAs.
How do I adjust the p values to control alpha error rate?
If I chose Bonferroni: Do I simply divide 0.05/10 and use the new value of 0.005 as the critical value? Or do I have to divide by 30 (because 10x main effect of time, 10x main effect of group and 10x interaction effect?)
I would much rather use Bonferroni-Holm, what would you recommend?
Context: Intervention group and waitlist control group, Baseline and Post Measures, Outcomes: Well-being, optimism, depression, stress, worrying etc.
Hello Celine,
You may, of course, use whatever adjustment you deem reasonable for the data set and your specific research aims. I'd lean in the direction of grouping the like effects and adjusting (e.g., the 10 main effects of "Factor A"). The ordinary Bonferroni is a wee bit conservative, compared to some other choices.
For executing your chosen adjustment:
Have a look at the R function, p.adjust. It merely requires entering a vector of observed p-values, with the desired adjustment method, and applies the identified method for you.
Here's the explanation, with example: https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/p.adjust
Good luck with your work.
In addition to Prof Morse I am attaching a paper that contains the Holm procedure. These tests are all available in R package bioconductor and I would recommend them for most uses. The R instructions are in the forward. If you need introduction to R I recommend Jared Lander R for Everyone available from the z-library. Best wishes, David Booth
You need to decide the "family" of test. You should do this before conducting your research since it affects the paper. As David Eugene Booth says, Holms "dominates" Bonferroni's meaning it controls Type 1 error as well, and is always at least as powerful. And it is the default the R function p.adjust, which also allows you to use Bonferroni's if you want (Holm's is sometimes call a modified Bonferroni). One important thing to remember, particularly if you have a lot of p-values (beyond whatever value you think p-values are), is that you need to decide how much your adjustment should affect Type 1 and Type 2 errors for the individual comparisons and the family. This will depend on the research question(s) that you have.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA