May be sequencing, blasting (cheking for homologous regions) and design of new primers (that exclude the homologous regions) can help you to avoid unspecific amplification.
Or just redesign of new FWD- and REV-Primers and checking for certain combinations, that do not amplify unspecific fragments.
As a general rule, increasing annealing temperature helps get rid of this kind of problem. However, it is hard to know without looking at an image of it.