I have sequenced one mitochondrial locus and computed pairwise FST‘s for all pairs of 7 populations using Arlequin. One of the reviewers asked me whether I had corrected p-value after multiple comparisons. But as I understand it, the correction is needed if I have a set of independent tests, for example, a set of different loci. Should I really do any p-value correction for pairwise FST‘s having one locus?
A p-value adjustment is the adjustment of a p-value of a single significance test which is a part of an A/B test so that it conforms to the rejection region of an overall null hypothesis that spans a set of logically related significance tests. A p-value adjustment is necessary when one performs multiple comparisons or multiple testing in a more general sense like performing multiple tests of significance where only one significant result will lead to the rejection of an overall hypothesis. A multivariate test (A/B/n test) is one such example. If one simply performs pairwise tests and is willing to abandon the control group based on a single statistically significant result, then an adjustment is necessary (in this case the Dunnett's Correction).
Please see the link for details: https://www.analytics-toolkit.com/glossary/p-value-adjustment/
Hello Natalya,
I suspect that the reviewer was concerned with the usual admonition that aggregate risk of Type I error for a set of hypothesis tests is somewhere between pc_alpha and 1 - (1 - pc_alpha)^c (where c is the number of comparisons made in total, and pc_alpha is the per comparison alpha level used). The upper end of that range assumes independent tests, and the lower-end assumes complete dependence among the tests (e.g., same test run c times). You're correct that dependence does reduce the aggregate risk (of any Type I error), but it doesn't reduce the expected number of Type I errors in a set of tests.
If you elect to go with some adjustment, consider one more powerful than the usual Bonferroni adjustment, such as the Benjamini-Hochberg. If you elect not to invoke an adjustment, then an argument such as better power (in the case of small samples, for instance), or of treating the analysis as exploratory might be more persuasive than relying on some indeterminate degree of test relationships.
Good luck with your work.
Hi well Steen Knudsen's series of books on Microarray Analysis in Cancer Research as well as nucleic acid research in general like chromosome studies is worth consulting on a topic like this. They are all available through the z-library google il, if you have a second. It is hard to understand that mendel's law on independent assortment was about genes on different chromosomes. It isn't too hard to follow reasoning that if all mutons are at one locus they are indeed not independent. I suggest you follow Knudsen's advise as well as Prof Morse's advice from a more statistical perspective. They both give the same advice. I attached a detailed set of instructions on these suggested methods. All of the advice is the same so please follow it. With best wishes for for a successful genetic research career, David Booth
Dear David Morse, David Eugene Booth and Maurice Ekpenyong ,
Thank you so much for the answer my question.
But I'm still confused. Could you explain more specifically? Here's the situation:
I have one single mitochondrial locus.
I have two populations (100 individuals each) which include 20 different haplotypes in total.
I’m computing Fst using Arlequin software. I’m new in population biology and statistics, but as I understand, Fst is calculated from average genetic distances between individuals within and between populations. And this genetic distance seems not to be calculated for each haplotype or mismatch independently. So I cannot understand, whether there are any independent tests in this process.
If I would want to do for example Bonferroni adjustment, I need to know the number of tests to correct my p-value. What should I consider as number of tests? The number of haplotypes? The number of variable positions?
When someone compares two populations using 10 loci, he uses Bonferroni adjustment just dividing 0.05 (or 0.01) by 10 and gets a new p-value threshold. And he never cares about the number of alleles in each locus. Or he should care?
Actually I have no problem to adjust my p-value threshold dividing by the number of haplotypes. My differences remain to be significant in such case. As I understand Bonferroni adjustment is the most rough method, so I should not expect that any other method turn the differences to insignificant. I just need to know must I use any adjustment and what number should I consider as a number of independent tests…
Thank you in advance!
Hello again Natalya,
I think the most straightforward option would be to identify how many comparisons are of interest to you in addressing your research aims, and use that as the target number of tests.
There may well be a "standard practice" in this field, but it's not my domain. I was trying to address the statistical implications of multiple tests.
Good luck with your work.
Have you tried looking at some of the references? If all of your genes are at the same place they are unlikely to to have independent statistical tests. After all the modified central dogma is true. Right? Basically the hypothesis of your question is backwards. You apply the correction for the non independent case to approximate the independent case not the other way around.. look at some references please?
BTW you have used the would independent incorrectly. Independent t test means the samples are independent. If you sample at the same place how likely is it that the samples are independent? That's why you use the correction. D.Booth
BTW in my last response it would have been more clear i had added after the ? If the samples are not at independent loci how can the test results not be related? Apologies for the fact that the word independent is used multiple ways in mathematics. Consult The Mathematics Dictionary published by Van Nostrand for more information on that Best, David Booth
- Badges
- Science topic
- Similar topics
- Engineering
- Computer Engineering
- Computer