Yes, there is a long list of studies on the effects of uneven sampling, amount and depositional setting of rock preserved from a certain geological time slice, so-called "lagerstätten effects" (peaks in diversity due to exceptional preservation), effects of better preservation of and knowledge about younger to recent groups ("pull of the recent") and so forth. See e.g.

https://www.researchgate.net/search.Search.html?query=fossil%20record%20%22sampling%20bias%22&type=publication

These questions came up in the 1980s when researchers became more active in their attempts to measure palaeodiversities. Some methods from "neontology", such as rarefaction curves, have been adopted.

Thanks! It's good to see a paper on ichthyosaurs in that list--I was thinking they would be a fantastic resource for this, and it confirms that I was at least on the right track.

I'm not so much interested in understanding the degree to which our sample represents the ecosystem. I'm more interested in knowing whether or not a formation has been adequately sampled. The question stems, as I said, from paleontological resources monitoring/mitigation legislation--many regulations I've come across call for preserving scientifically significant fossils, and most of the time that de facto ends up meaning vertebrates (minus fish). I think a reasonable argument could be made that in poorly-sampled outcrops even invertebrates may be significant. That legislation is all based on the geological formation.

My thinking is that if we treat monitoring as a sampling effort within a larger historic context, we can make a more accurate determination of the probability of even invertebrate remains (macro and micro) contributing significantly to our understanding. In poorly sampled formations any new information may be significant; in well-sampled formations additional sampling isn't likely to tell us much, and the invertebrates are less likely to be significant. While this may seem harsh, right now invertebrates are largely written off in this work, so even this would be a substantive improvement.

The biggest problem we have is the hollow curve distribution. It is effectively fractal, in that you will never find everything in the deposit; the more you have sampled, the closer you come to seeing the rarest taxa that are present, but the abundance curve will look much the same whatever scale you are working on.

We have found over many years that even sites that appear to be well known have only been lightly sampled, and new species appear regularly with more collecting... so it depends on your definition of 'adequate'. Perhaps one approach will be to consider what volume of new samples would be needed in order to start expecting new taxa? That can be assessed from the size of previous collections and the diversity of species recorded. Chao estimates can sometimes be really useful here, as you don't need full collection data, provided you have the basics about two or more large-ish collections.

As Mason says, the palaeoenvironment (and also the range of sediments at the site, and other factors) have a strong influence on the total that you can expect. Aberrant communities with inherently low diversity (e.g. lagoons) are unlikely to keep producing new taxa indefinitely. Entirely homogeneous, rapidly deposited rocks with no variation in faunas through the section are also easier to approach the maximum diversity of. Take a thick, condensed, or very varied section of mudstones and sandstones, with some degree of exceptional preservation in at least some layers, and it will keep producing new stuff until you've quarried out the entire bed.

The other thing to consider is collecting technique. Microfossils, for example, will regularly produce new taxa if prepared in a slightly different way, or looked at by specialists in different groups. On the other hand, if microfossil samples have been examined and found to contain only a depauperate, poorly preserved assemblage, then you can probably not worry too much about them.

There are too many variables for any universal metric to do what you want it to, I'm afraid. You're going to have to look on a case-by-case basis, and assess each one qualitatively, and I doubt whether anyone has put together anything like a comprehensive guide. Which means, of course, that there's a gap for you to fill! :o)