What you need is an image segmentation process such as that in Adobe Photoshop. This can be used to map your image RGB pixel values onto CIE L* a*b* colorimetric co-ordinates and L*C* H* (lightness Chroma and Hue) values. We may now segment the pixels in any given part of the image into successfully identified and isolated clusters that can be altered in Hue or Chroma or Lightness (or any L,C or H combination) by iterative optimization. I have in the past use this approach to model the effect of surface texture on colour appearance, to isolate and recolour the lips in a facial image or turn the hair blond (see my RG posts ‘ICHT Figs’ and ‘Measuring the Contribution of Texture to Colour Appearance’. The cluster limit definition may be either by maximum colour difference CIE DE from the chosen mean or by a single co-ordinate DL DC or DH difference. In practice I found DH (i.e. Hue difference) was the most successful cluster isolation parameter.