Suppose individuals always mate exclusively within their phenotype group for a single gene. Can Hardy-Weinberg equilibrium formulas still be used to predict allele or genotype frequencies for this particular gene?
When we restrict the mating partners in a population, it goes against the condition of random mating, as assumed in Hardy-Weinberg equilibrium. So, ideally, we should not apply Hardy-Weinberg formulas here.
You would start by using Hardy-Weinberg to show the the population IS evolving (and must be violating one or more of the assumptions of the model). You can treat "homozygous recessive" as if it's a lethal selection (since the individuals with that genotype & phenotype are being removed into their own population). Add in selection against the recessive allele and you'll be able to see how many generations until the population reaches fixation.
Hardy-Weinberg equilibrium tests precisely for non-random mating. This is rarely achieved in nature. You already know that the mating is not non-random. So I am confused about why you would invoke HW. The null hypothesis for HW is that mating is random. The extent of the deviation from randomness measured by the p-value should give you a sense for how non-random they are.