That depends on the color values in your matrix. It most likely is an int32 with 3 byte color info followed by 1 byte alpha channel. The color info may be in RGB or BGR order. You can simply extract the values by bitshifting.
You can use MATLAB Image Processing toolbox for this purpose. The command you are looking for is "impixel(arguments)". This command can return the Red,Green and Blue values for any pixel that you select. For further details it might be good idea to look at the Help menu in MATLAB.
If you have a 3 dimension image matrix you have in each dimension the value of the pixel for one channel, if you are more specific in your question I can give you more details, but all depends in how you extract the matrix from the image.