I am interested in comparing mean protein levels between 4 groups (Control, Stages 1-3), after controlling for age, gender and a genetic risk factor. My pre-specified comparisons are : Control vs Stage 1, Control vs Stage 2, Control vs Stage 3, and Stage 1 vs Stage 2. I attached a picture to help explain my goal.
I plan on running this analysis for 45 proteins, which have been specifically chosen. Therefore, each protein will have 4 p-values. Ideally, I am interested in adjusting the p-values by using the FDR Benjamini-Hochberg Procedure. Should I adjust each type of comparison (i.e. control vs stage 1, within a column) by 45 tests? A recent suggestion was to adjust the p-values for a given protein by 4 tests (within a row). Finally, I believe adjusting for the total number of tests 45*4 = 180 would be too conservative.
Any suggestions would be appreciated. Thanks !
Hi,
If the given number of proteins are all ranked then the post hoc comparison which you mentioned is useful. See the link below:
https://www.youtube.com/watch?v=rZKa4tW2NKs -For FDR BH procedure.
If you are only looking at the Row comparisons which are 4 in the table you have used then the Post Hoc Bonferroni correction is apt. Divide alpha by the number of tests.
https://www.youtube.com/watch?v=HLzS5wPqWR0 -For Bonferroni correction.
It seems to be a screening across 45 genes and 4 contrasts. In this case a correction should consider all 180 tests that are performed.
Note that "being conservative" is not related to the number of tests you correct for (this is just accounted for by the multiple testing correction!), but to the FWER or FDR you are willing to tolerate. If 10% FDR is too conservative for you, then use a 20% FDR.
Ideally, in such experiments you should not dichotomize between "hits" and "misses" or between "candidates" and "non-candidates" based only on a p-value. You can use the estimate together with the standard erros or p-values to rank the list from the "most promising" to the "least promising" candidates to be tested in a following, independent experiment. As confirmatory tests, those would not require adjustments.
And after all, the real story begins where you show experimentally that the gene products (usually proteins) are affected and that the gene regulation (over-expression, knock-out/down) imapacts the biological response in-line with the conjectured model.
No, you include all of the data in one big file and then obtain one p value from your ANOVA or PERMANOVA and then perform post hoc tests if p is < 0.05. If you do separate tests, you miss interactions and increase the risk of Type 1 errors :)
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression