Hello community,
I have experimental designs with two independent variables (IVs) and performed a two-way ANOVA to check for the individual and combined effects. I have read that when interaction isn´t significant is not necessary to do post-tests because the information is already in the IV results. But when interaction is significant, I would like to know if it is right to perform a t-Test with 0,01 of significance as post-test for certain pair of comparisons (4 comparisons in total) and avoid using post-test. In this way the idea is to gain more statistical power maintaining the Type I error in 5%, as post-tests do.
Thanks!
No because of the problems of multiple comparison. For a discussion and a better approach see the attached. Best, David Booth BTW your t-tests could be used with the suggestions in the attached. For example, see: https://www.amazon.com/Cancer-Diagnostics-Microarrays-Steen-Knudsen/dp/0471784079/ref=sr_1_4?keywords=Steen+Knudsen&qid=1582925310&s=books&sr=1-4
Hi David, thanks for your response. As I could get from a post the formula for the Experimental Error Rate for an entire set of comparisons is :1 – (1 – α)^C. Considering that if I do just 4 comparissons with a significance level od 0,01 for each one, I would get a EER of 4% that is still less than the 5% error that post-tets (Tukey, Bonferroni, etc) use. So, I still don´t understand why is it wrong to perform selected comparissons with a T-Test after an ANOVA to increase the statistical power.
Best, Andrea!
Because you must correct for the fact that you are doing multiple tests on correlated values. The test you selected was not incorrect you just need to adjust those results for the multiple testing situation. HTH. Best, David Booth BTW if you did only one t-test-no correction would be needed
Oh, now I get what you mean. Thanks! In the file you uploaded Bonferroni, Holm-Bonferroni, Benjamini-Hochberg are suggested in R, but I have my data in Prism which doesn´t have those tets. Instead, it offers Holm-Sidak. What is the statistical power compared to the ones mentioned above? Would it be ok to use it anyway?
Andrea Portales Yes I think so but notice you can do the Bonferroni easily by hand. The formula is M*P where M is the number of tests and P is your current p-value. Suppose your current p-value is 0.01 and your current alpha is the usual 0.05. and you do 2 t-tests. Then M*P=2*(.01)=.02
Andrea Portales I perhaps should mention that I almost always use Bonferroni because it is so easy to use and remember. My statistical mentor used to say it's more important to use a method than exactly which method you use. So I always use Bonferroni because I can remember it unless I have a particular reason to choose something else. For an example of a particular reason look up Bonferroni in this link:
https://b-ok.cc/book/736739/f02468
Best, David Booth
David Eugene Booth should I report the corrected p-values or the significance level used for the whole family of comparissons?
Both the corrected p's and your overall level which is still 0.05 say you used Bonferroni and reference the booklet I sent. Best wishes, David
- Badges
- Science topic
- Similar topics
- Geoscience
- Asia
- India