Regarding your question if you see the medium composition, it contains sucrose as carbon source and bromothymol blue and thymol blue pH indicators, Vibrio which utilizes sucrose as carbon source and produce yellow colonies and non sucrose utilizing produce blue green colonies on TCBS agar.
I recommend you to sterilize by filtration (0.2 um) the sucrose and also to use sucrose grade HPLC 99 % of purity or higher to be sure you don't put any inverted sucrose in the TCBS medium. I think that minimal levels of Glucose and/or Fructose could give you yellow colonies, but not due to the breakdown of sucrose by the microorganism.
The acidification of the medium resulting from the fermentation of sucrose by Vibrio makes bromthymol blue turn yellow. Bromthymol Blue and Thymol Blue are pH indicators. ... An increased pH is used to enhance growth of Vibriocholerae, because this organism is sensitive to acid environments.