You do not give much information to go on, but there are a number of dye based cell enumeration assays that are used to determine the cytostatic or cytotoxic effects of different compounds.
The use of a surrogate dye is already a compromise as you are really interested in the number of cells present after treatment and whether the test compounds are causing cell death or not. You also want to know how selective the test compound is toward the cancer cells relative to rapidly growing healthy cells.
To be able to see small differences you would need to use a spectrophotometer. The test compound should be tested at different concentrations to determine a dose response curve, and this is not an all or nothing approach.
Just to give you some idea the following assays can be used that give colored derivatives depending on cell numbers in the culture wells.
SRB, MTT, Nile red staining, GFP incorporation, Crystal violet, Alcian blue etc
Perhaps you should find a good review article and see which would suit you best.