The neighbour populations belong to the same genetic pool, however they are morphologically differents, private alleles could be the explanation to an ancient selection of an adaptative shift?
just looking at private alleles doesn't tell you the whole story. is polymorphism lower? there could just be a bottleneck (independent of adaptation). can you calculate Tajima's D?
As already mentionned, it's hard to conclude about differences of private allele number with so little information.
To add to the other answers, this could also be explained by asymmetrical gene flow between your populations, if there is gene flow of course (see attached ref).
Article Testing for Asymmetrical Gene Flow in a Drosophila melanogas...