I would like to interpret a trained model for an image recognition system and understand which part of the feature vector(the last layer that I am using for cosine similarity or matching with enrollments) is for color information.
For example, if I have two images like this:
I want to know which part of the feature vector is different and this difference is related to color differences.