You could try measuring the antibacterial activity of trace concentrations of the solvent under the same conditions. If you also can get a measurement of the solvent concentration in the sample (talk to an analytical chemist about that), you would then be able to tell whether the amount of solvent present in the sample is sufficient to account for the antibacterial effect of the extract.