This is a very good question. Based on resent findings about the multiple ecological and physiological functions of tannins and their different modes of action (see publications by Salminen et al.) it makes most sense to analyse them compound by compound (e.g. not in bulk) through LC methods (HPLC, LC-MS). This does also allow to compare relative ratios of certain tannins to others as well as the use of tannin compound diversity as a variable that may vary with adaptations to certain ecological functions.
A cheap way that doesn't discriminate between various types of tannin is a radial diffusion assay, where you embed protein into an agar matrix, punch a well in the agar, and dose some leaf extract into the well. As the tannin binds to the protein, it forms a precipitate. The diameter of the ring is then proportional to the amount of phenolic in the leaf and gives you an idea for how reactive that tannin might be in the gut of the herbivore.
You may attempt to calibrate a NIRS or MIRS procedure with a set of known samples previously analyzed by conventional methods. Once the method is calibrated you can run hundred of samples per day. It is the most ecological way of doing it. No reagents, no sample preparation ,etc. You may even take measurements directly in the field if your calibration is performed with field moisture samples and accounts for moisture is given.
This is a really good question. It would be fantastic if someone could formulate predictions for the system plants/big mammals. I'm involved in a project comparing structural and chemical defences in plants. I've asked the same question to different specialists of chemical defences and I now have a growing list of variables to test (one compound=one variable, same for NIRS peaks, same for ratios,same for tannin compound diversity).
My problem is simple: as my number of variables is increasing, my multiple testing correction (Bonferroni-like) is getting more and more restrictive (angry). It would eventually lead me to miss the important variable. To avoid this general problem in biology, I need good predictions involving a restricted number of variables.
If somebody could provide biological hypotheses linking one specific compound concentration/ratio/index of diversity with a deterring effect on big mammals, it would really be helpful.
The best to my-opinion is the PEG-binding assay, possibly by NIRS.
Please read:
DeGabriel, J. L., Moore, B. D., Marsh, K. J., & Foley, W. J. (2010). The effect of plant secondary metabolites on the interplay between the internal and external environments of marsupial folivores. Chemoecology, 20(2), 97-108.