You may prepare the stock solution of the antibiotics in DMSO. The stock solution can then be further diluted in culture media to prepare a working solution. Make the stock solution in such a way that when you dilute the stock, the final concentration of DMSO to be used in culture is less than 0.1%.
0.1% DMSO as the final concentration in culture is considered to be safe for almost all cells.
For instance, 100mM stock solution of antibiotic prepared in 100% DMSO when diluted in culture media by 1:1000 will give 0.1mM working solution which will contain 0.1% DMSO which is non-toxic to cells.