(Sorry I am going to multi-post here so my replies aren't as hard to read)

First, do you have a negative control specimen (same human sample type without aspergillosis) to run the multiplex PCR on? If you do, that will answer pretty quickly if your primers are just binding to non-specific sequences in the human genome (or other microbes, depending on if the sample site is typically sterile or not).

Second, if you do not have access to such a negative sample, then I would

use Primer BLAST (https://www.ncbi.nlm.nih.gov/tools/primer-blast/) to check for specificity of your primers against the human reference genome to see if they can create unintended products on human DNA. The tool is meant for designing primers de novo but also has an option to use pre-designed primer sets to check for specificity against a selected genome or set of genomes.

If you have not used this tool before, you will need to copy and paste your primers into the "Use my own forward primer (5'->3' on plus strand)" and "Use my own reverse primer (5'->3' on minus strand)." You can ignore the other primer parameter options, I believe Primer BLAST ignores all those settings once you enter a custom forward & reverse sequence.

Since you're multiplexing, you will need to run the tool multiple times depending on how many primers you have in different combinations, in other words if you have 4 primers you will need to check the specificity of not only primer 1 + 2, but also primer 1 + 3, primer 1 + 4, etc. until you've covered all possible combinations. Maybe someone knows how to do this automatically but I only know the brute-force way.

Under "Primer Pair Specificity Checking Parameters" make sure the "Enable search for primer pairs specific to the intended PCR template" box is checked (it should be by default anyway) and change the database from Refseq mRNA to Refseq representative genomes. Make sure "Homo sapiens" is filled in under the "Organism" box. I would also change the default values for "Primer must have at least [2] total mismatches to unintended targets, including at least [2] mismatches within the last [5] bps at the 3' end." So that it's within the last [7] bp at the 3' end.

Click "Get Primers." Once it runs, it should either say that a primer pair is specific to the template and that no off-target effects were found, or will give you a list of potential unintended targets. If you do get unintended targets on the human genome, you can compare the predicted PCR product sizes on the unintended targets to the unintended targets on a gel.

It may only require a small redesign of a couple primers to prevent non-specific products.

Third, if human DNA does not seem to be the issue then I would probably just extract gel extract and either directly sequence, or clone then sequence the non-specific bands you're getting. Then you can BLAST the sequencing results against any known database to get an idea of what's going on. From there, you can start to formulate a plan to redesign your primers or change something in your extraction procedure, if the goal is to directly multiplex PCR human aspergillus samples repeatedly. I could definitely see scenarios where small amounts of genomic DNA from related fungi are contaminating some reagent being used and getting partly amplified when the amount of target DNA is low, but without knowing what the sequences are that's total speculation.

Thank you for your precious suggestions