I am consistently getting higher baseline Fst between populations when one of them has a lower population size (24 vs 48). When Fst is calculated as a derivation of pi and dxy:

0.5*(pi_1+pi_2)/dxy

Then that elevated background signal disappears. Outliers remain in the same locations, but the magnitude is diminished. Is this some sort of a quirk of how Weir and Cockerham correction for low sample size is? Has anyone had this issues?

Similar questions and discussions