Which part of rosemary you used. if you used the leaves so the problem is chlorophyle, usually it is not easy to remove the colour because very dark. You can use coloumb Chromatography to remove the chlorophyle, and you will get the clear extract.
The answer depends what you mean by phytochemicals and what you final objective is. The colour in extracts is caused by plant metabolites (phytochemicals) that are coloured.
There are destructive methods such as adding a bleach e.g. hypochlorite or leaving extract in sunlight to cause photooxidation. As colleagues have stated above if you have leaf extracts colour is usually caused by chlorophyll or its decomposition products. I am not sure exactly how it works but I have seen colleagues using reverse phase Sephadex LH20 when the highly non-polar chlorophyll derivatives are retained on the bed.
In my own experience the chlorophyll compounds are very close to the solvent front in TLC using Silica gel. A solution could be to use a short Silica gel column and then elute the column with an intermediate polarity solvent such as ethyl acetate or acetone. The colour will come out very quickly and the other plant metabolites can be obtained separately by eluting with a more polar solvent such as methanol.
I know many phytochemists "defat" plant material by extracting with hexane and discarding this extract. This extract should contain most of the coloured compounds. I am very wary of this approach because we have several examples where the most active compounds were present in the hexane extract.
A techniques retaining (mostly absorbing) color (color may be because of colored constituents which could be secondary metabolites, degradation products or other high molecular weight colored products as well as products of other origins like intermediates, elicited or induced products) or, a combination of products generating a color is supposed to be good. The filtered out material may produce less coloration in the final extract (you may need to concentrate it again!) or nearly no color depends on various factors and presence of constituents including the type of plant (higher plants or Bryophytes, Gymnosperms, Angiosperms plants, fungi, mushrooms etc.), plant parts used, extract solvent (solvent used to extract material) and concentration of the extract (richness and abundance of the constituents). You need to choose and work carefully. Some loss of constituents will be always there. The question is what are your constituents of interest? For low molecular weight products, you could use the technique but avoid throwing the Hx (Hexane) extract as some secondary metabolites may come associated with the filtered extract due to similar lipophilicity levels of the extract and/or a combination/ molecular association with other products, and you may lose it, as in the case reported by Prof. Eloff.