Can you elaborate on what you have done already? What do you mean by the right level? When you test a solution of known concentration do you not get close to the stated concentration? Is it over-recovered or under-recovered?
You haven't specified the equipment you are using, so I'll suggest a few things.
Mercury and silver can tend to be "sticky" in the uptake tubing, particularly if there is chloride present. Consensus is either no HCl or lots of HCl is needed in the matrix.
What is the measurement of uncertainty of your method? If you get 10 ppb ± 1ppb, is that good enough for your purposes? if it is 1ppb ± 50%, is that not suitable?
How did you calibrate these metals? (What acids are present) Is the sample solution treated with the same acids? If you use 1% nitric acid to calibrate, make sure the sample also has 1% nitric acid.
Look up the following link:
Silver Chemical Stability https://www.inorganicventures.com/guides-and-papers/silver-chemical-stability
To improve ICP-MS mercury determination in drinking water when observing increased recoveries, add gold as a stabilizer to prevent mercury loss, optimize sample preparation (acidification and filtration), and adjust plasma and interference settings for better accuracy.
Are you evaluating the recoveries on a fortified matrix? If so, have you checked the presence/absence of W? Because it forms WO, which have mass 200 and will interfere with Hg; if you have it inside your matrix, try to use the reaction cell mode with O2, which shift the mass of mercury to 216/217 (the side effect is loss of sensivity). For Ag, prepare daily your standards, and (as mentioned above), use the same concentration of acids for the calibration curve, internal standards (if you add them online) and samples.