I've been thinking about this lately, but haven't been able to properly wrap my head around it. I recall that the prime-corrected F-statistics (and Fst analogues) were formulated because Fst is limited by the average subpopulation heterozygosity (Hs) such that the maximum possible value of Fst is 1-Hs (Fst(max) = 1 - Hs). So G'st accounts for this and normalizes the value by dividing Gst by Gst(max) (in other words, G'st = Gst/Gst(max)).
This makes intuitive sense, especially in situations where there are lots of alleles; subpopulation heterozygosity can be high, even if two populations have no alleles in common, and Fst fails to reflect this.
But the statistic is framed and defined by heterozygosity, not the number of alleles. So G'st can still produce values significantly larger than Gst even when there are only two alleles (as with many SNP datasets).
So my question is: is the G'st statistic (and, by extension, G''st) appropriate for biallelic SNP data? Or are they simply not intended for this particular case?
G(ST) and its relatives are often interpreted as measures of differentiation between subpopulations, with values near zero supposedly indicating low differentiation. However, G(ST) necessarily approaches zero when gene diversity is high, even if subpopulations are completely differentiated, and it is not monotonic with increasing differentiation. Likewise, when diversity is equated with heterozygosity, standard similarity measures formed by taking the ratio of mean within-subpopulation diversity to total diversity necessarily approach unity when diversity is high, even if the subpopulations are completely dissimilar (no shared alleles). None of these measures can be interpreted as measures of differentiation or similarity. The derivations of these measures contain two subtle misconceptions which cause their paradoxical behaviours. Conclusions about population differentiation, gene flow, relatedness, and conservation priority will often be wrong when based on these fixation indices or similarity measures. These are not statistical issues; the problems persist even when true population frequencies are used in the calculations. Recent advances in the mathematics of diversity identify the misconceptions, and yield mathematically consistent descriptive measures of population structure which eliminate the paradoxes produced by standard measures. These measures can be directly related to the migration and mutation rates of the finite-island model.
above may help you
Hi Hemanth Kumar Manikyam ,
Thanks for your response! I have read Jost's paper (and many both before and since) comparing these methods and statistics. Unfortunately, I don't think it actually addresses my question.
- Badges
- Science topic