About minerals observed in thin section by optical microscopy under transmitted plane polarized light, you may find a clear explanation of birefringence effect and related polarization colours at: K.G. Cox, N.B. Price, B. Harte, "An Introduction to the Practical Study of Crystals, Minerals, and Rocks", rev. 1st ed., McGraw-Hill, 1988, Ch. 6. Microscopy by reflected polarized light has been preferred in optical metallography: cf. (e.g.) A.F. Hallimond, E.W. Taylor, Ch. 3 in: G.K.T. Conn, F.J. Bradshaw, "Polarized Light in Metallography", Butterworths (London) / Academic Press (New York), 1952. About polarization colours you may also refer to Ch. 6 at: P.C. Robinson, S. Bradbury, "Qualitative Polarized-Light Microscopy", Royal Microscopical Society / Oxford University Press, 1992.    

Many thanks Dr. Carlos

The full wave plate can help to understand where the colors come from: https://micro.magnet.fsu.edu/primer/techniques/polarized/firstorderplate.html

Basically, a birefringent material changes the polarization of light.  However since the polarization change is due to a phase change between the ordinary and extraordinary waves, and different colors have different wavelengths, this means that different colors will undergo different degrees of phase shift as they pass through a birefringent material.

The above link is just for the full wave plate, but the site also has a full discussion on birefringence and polarized light microscopy:

https://micro.magnet.fsu.edu/primer/techniques/polarized/polarizedhome.html

 In order for a birefringent crystal to produce interference colors in polarized light, it must retard electromagnetic radiation in the visible portion of the light spectrum. If there is no retardation, then a birefringent crystal appears white between two crossed polarizers. Retardation means that at least one wavelength of the visible light is rotated such that it remains parallel to the light vibration plane coming out of the polarizer. When a second polarizer is placed at 90 degrees to this plane, the waves that exit the crystal that are not perpendicular to the second polarizer (the analyzer) are permitted through the analyzer and the waves that are perpendicular are blocked (retarded). The color that is seen will be the complementary color to the predominant wavelength blocked (retarded). A full wave plate is usually a crystal milled to retard green light (550 nm) be permissive for all other wavelengths, and in a polarized system the entire background will look red-magenta as a result. This plate is used to distinguish between the two most common types of biological crystals (monosodium urate monohydrate and calcium pyrophosphate dihydrate) seen in human specimens because while both of these crystals are birefringent, both also pass all wavelengths of light in a polarizing system( they retard waves in the 100nm range)--so both will be white on a dark background. However, since the ordinary and extraordinary components of birefringence are exactly the opposite of one another in these two crystals, adding a waveplate can distinguish between the two, because 100 nm, when additive with 550 nm will retard light of 650 nm (red-orange) and the crystal will appear blue--the complementary color. When 100 nm is subtractive, light of 450 nm (blue-green) will be retarded and the complementary color of the crystal will be yellow.

If there is no retardation, then a birefringent crystal appears dark (not white, as it was mentioned above) between two crossed polarizers. When the retardance (= birefringence x thickness) increases the passed beam exhibits the Newton’s interference colors. The hue is determined by wavelength, which is missing from the spectrum due to destructive interference. For small retardance (

Pallab Kumar Borah

Would suggest in reading this article as it may be helpful for you

https://www.olympus-lifescience.com/en/microscope-resource/primer/lightandcolor/birefringence/