Correct me if I am mistaken.
Simply put, my study involves doing RT-PCR for 13 different genes in a Two-Way ANOVA model. When I attain an output for my Two-Way ANOVA, I get an interaction effect, and two main effect p-values (2 x 3 TWO-WAY ANOVA).
To correct for false positives, I run a Bonferroni's adjustment after my Two-Way ANOVA to determine if my interaction, or main effects are significant. Considering I have 13 genes, I multiply each p-value by 13 to attain a q-value. If q is significant, then I would run the appropriate corresponding statistical analyses. I have two main concerns:
1) Where I am stuck on is when my q-value is significant for my interaction effect. My statistical software gives me a p-value but I cannot put in the Bonferroni adjustment that it should be also considering. So I manually multiply each given p-value by 13 to attain my q-value. Unfortunately, I do not know how to make the appropriate adjustments for my computed F values. Can anyone help me with my situation? How do I also adjust my F values for Bonferroni's.
2) If I find that either my MAIN effects are statistically significant after Bonferroni's adjustment, would I just run a regular post-hoc analysis (Tukey's) for each main effect? OR would I also have to multiply each post-hoc analyses output by 13. I already have to do a Bonferroni's adjustment for my independent variable with 3 levels (i.e. multiply each p-value by 3), but do I also consider the 13 genes here as well? Simply, do I use the regular computed p-values AFTER Bonferroni's adjustment for my MAIN effects.
I know this is pretty complicated wording so if there is any additional information I can give, please feel free to let me know.
If I am approaching this incorrectly, please feel free to give me any advice on what I should be doing. Any suggestions or comments are very well appreciated. Thank you!
Dear David,
I am a bit puzzled by your questions and some things remain unclear to me (maybe because I am not a biologist), so I do not know what a RT-PCR for 13 genes is. What does it mean for the analysis? Why did you correct for 13 genes? Did you run the 2x3 ANOVA 13 times, so one for each gene or how is this to be understood? Why are you using q-values? I only know them from False Discovery Rate (FDR) calculations, which might be more reasonable that Bonferroni, since it is less conservative.
But maybe some general information which might help. If you want to use Bonferroni as a correction, than you only need to calculate the new/adjusted a priori alpha-error level. So, typically a 5% alpha-error level is used. If you want to correct for 13 comparisons (we assume that this is correct for the sake of argument) you only need to divide the original alpha-value by 13, i.e. 0.05/13 = 0,0038. Now you compare the p-values from your analyses with this adjusted alpha-level, i.e. with 0,0038 or 0,38% instead of 5%. If your p-value falls below the new level you consider it as "significant". No need to adjust any F- or t-values.
For how many comparisons you should adjust is hard to say, I think there are several opinions about it, but please have a look at the attached pdf for a more conservative approach.
If you use post hoc test such as Tukey, it already considers a familywise error rate, in that case you do not have to adjust it again.
Hello Rainer
I apologize for the delay in response. Regarding my study, to try to put in general terms that would be understandable, I have 6 groups in total; 3 groups are categorized into a subset of 2 (i.e. A1, A2, A3, B1, B2, B3, where 'letter' denotes different types of vitamins - 2 vitamins - and 'number' denotes doses in each vitamin - 3 levels. My "RT-PCR" is just a single dependent variable, so consider them any continuous dependent variable. I have 8 samples into each of the 6 groups (n = 8/group). I then test for significant effects from each independent variable (diet - A v B - and dose - 1 v 2 v 3) by Two-Way ANOVA - barring no interaction effect. Post-hoc's are done by Tukey's for diet or Bonferroni's for dose. I am running 13 different Two-Way ANOVAs (i.e. 13 different dependent variables), BUT I am not comparing BETWEEN Two-Way ANOVAS (i.e. each of the 13 dependent variables are not going to be compared to each other). In this case, would I still use 0.05 (alpha) / 13 genes = 0.00384?
I have an additional question to the aforementioned information. SAY I am doing this for two different tissues (i.e. small intestine and liver), BUT I am not comparing between the two tissues in any way. Is there any further adjustment I would have to make for this? Since I am doing two tissues, I am running 26 Two-Way ANOVAs, so would I really have to do 0.05 (alpha) / (13 genes x 2 tissues)?
Sorry for the confusion. Please let me know as soon as you can! I really need help with this
There is no simple answer to this question. How many type I errors are you willing to commit? And this question does not simply stop if you will not compare the results with each other.
From Wiki:
Hochberg & Tamhane defined "family" in 1987 as "any collection of inferences for which it is meaningful to take into account some combined measure of error"
When will it be meaningful? Maybe you can find an answer in the Lehmann (2005) article.
For example, you have 10 variables and you will correlate them with each other. You will have n*(n-1)/2 = k correlations, i.e. 45. Even if you dont want to comare them, it seems plausible to correct somehow. Maybe you should consider another correction approach like False Discovery Rate or something like this, because it is well known that Bonferroni massively reduces test power.
But please have a look at the attached articles.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software