I know that Prof. Gitelson has been doing studies on Chl and RS. You may want to check his list of publications below. Specifically, you may want to read "The peak near 700 nm on reflectance spectra of algae and water: relationships of its magnitude and position with chlorophyll concentration" (Gitelson, A., 1992).
My papers on chlorophylls are what you need for chlorophylls. Here is my EXCEL calculator. The papers are on Research Gate.
I have updated my 2008 Chl calculator and will be put up soon.
The 2008 version is OK except for the Chl d algorithm where the extinction coefficient got revised a few years after I published my paper in 2008.
There are rough formulae for carotenoids around but they are not wonderfully accurate. We used them in our recent paper on the seagrass Halophila.
Feb 2021 - I have recently published some papers on measuring Chlorophylls in vivo in J Applied Phycology. The algorithms work quite well. Frankly I was surprised how well they worked.
Feb 2021 - Carotenoids are a problem. Most photosynthetic organisms have lots of carotenoids. Those found in Green Algae are the same as in higher plants and so you can use the algorithms developed for higher plants. The carotenoids in most other algae are not chemically similar and so have different absorption properties and so the Higher Plant algorithms cannot be used. A Chl c1c2-Chl a-Carotenoid algorithm is very much needed for aquatic biology but no-one seems to have done it.
You can see my paper“Newly Combined Spectral Indices to Improve Estimation of Total Leaf Chlorophyll Content in Cotton”, which is downloaded in my researchgate. I hope that it will help you. Gitelson et al. have carried out lots of the related study in this direction.
Your question is not precise because you did not mention if you want to measure these pigments after extraction or you want to measure directly the intact cells in their original water sample.
Let's discuss first the extraction method: First you should choose the proper solvent. That is a crutial question because not all solvents can extract the pigments from all algae. I would suggest preliminary experiments with this problem. Hot methanol used to work but there can be lots of problems depending on the alga species. Acetone works sometimes but not always. It is good for higher plants but not always with algae. Ethanol gives dubious results; the efficiency of the extraction is questionable.
When you know the solvent, choose the proper wavelength values and the equations from the literature. Be accurate with this beause the absoption band positions and the extinction coefficient values vary with the solvents.
A totally different problem is if you want to measure intact water samples. After proper calibration, you may get results, but the main point is the calibration of the equipment. Since these equipments usually work on the basis of reflectance, the water quality can cause artefact problems. Anyway, this "native sample" measurement provides semi-quantitative results, in some cases not even comparable with the extraction method.
I agree with much of what you say. Chlorophyll extraction from algae can be a major nuisance. Porra has written a lot on "recalcitrant" algae. Some will not extract in acetone, methanol or ethanol even if dipped in hot water. They have to be ground up in sand or glass beads. Ethanol is a better extractant than you think but you need to use dry alcohol. Only a few % water results in micellisation. Acetone is almost useless as an extractant on some green algae. The exotic solvents DMSO and PMF will extract chlorophylls better than acetone, methanol and ethanol but safety officers do not like them and you may be banned from using them.
In vivo methods. A water PAM is very good for measuring Chl a by fluorescence but you have to calibrate it. For algal cultures it is possible to get a correlation between Chl a in vitro and absorbance at 680 nm of cells suspended in 60% sucrose. In case you do not know 60% sucross has a refractive index similar to cytoplasm and so a turbid cell suspension looks like a clear solution when the cells are suspended in 60% sucrose. Considering the variability of efficiency of extraction of Chlorophyll in solvents the in vivo measurement in 60% sucrose would probably be a better bet.