I would start by checking the specificity of your primers using a program like https://www.ncbi.nlm.nih.gov/tools/primer-blast/ . If your primers are as specific as they can be, try increasing the annealing temperature of your PCR reaction.
You will need to calculate the appropriate annealing temperatures based on the sequence of your primers and the buffer you use for PCR. You can use https://www.thermofisher.com/us/en/home/brands/thermo-scientific/molecular-biology/molecular-biology-learning-center/molecular-biology-resource-library/thermo-scientific-web-tools/tm-calculator.html to get an estimate of the annealing temperature or a calculator for the specific buffer you use may be available from the manufacturer.
From there you can also try the reaction at a few degrees higher if you need to increase the specificity.
After you have run a temperature gradient of primer annealing temperatures you might try running a DMSO gradient from zero to 8%dmso final concentration. This not only reduces secondary structure but increases the specificity of the primers so minimises inappropriate binding of the primers to other similar template targets. Use a sample that amplifies and amplify in 1%,2%,3% etc up to 8% dmso otherwise change nothing in the protocol. You should get increasing specificity up to the point where the pcr fails but it frequently works and the results are similae to an annealing temperature gradient