If you have a compound showing anti-cancer activity, then you will have to perform cytotoxicity assay such as MTT to check for its potency. Potency is an expression of the activity of a compound in terms of the concentration or amount of the compound required to produce a defined effect. When you perform the MTT assay on mammalian cells preferably cancer cell line using an increasing concentration of the candidate drug you will see an effect i.e., with increasing concentration of the candidate drug there be a corresponding decrease in cell viability.
Plot the graph of drug concentration on X-axis and cell viability(%) on Y-axis. Calculate the IC50 value from the graph. IC50 value is the half-maximal inhibitory concentration which is the most widely used and informative measure of a drug's efficacy. It indicates how much drug is needed to inhibit a biological process by half, thus providing a measure of potency of the drug. Lower the IC50 value, more potent will be your compound.
Having a compound showing anti-cancer activity does not mean that it has high cytotoxicity. You need to find its potency by performing the cytotoxicity assay.