Indeed monitoring programs often ignore imperfect detectability. Although many monitoring methods that account for imperfect detectability have been developed, still indices are largely used. For example, in the marine environment plot sampling (which assumes perfect detectability) is by far the most commonly applied method for biological monitoring. And papers based on plot sampling and relative abundance estimations (i.e. indices) are getting published all the time in high quality marine biology journals such as MEPS, Marine Biology, Fisheries Oceanography, Fisheries Research etc. Check the following paper for a recent review of the issue for marine populations:

https://www.researchgate.net/publication/230730079_Monitoring_marine_populations_and_communities_methods_dealing_with_imperfect_detectability

Article Monitoring marine populations and communities: review of met...

Thanks for the response and paper Stelios - sadly detectability is still ignored in the terrestrial realm too. I'm wondering whether to bother going to the effort to do things right, when journal editors seem happy to accept papers with clear inherent biases and ignored assumptions.

I agree that the index surveys are only of value if they have been conducted over long periods of time by the same people. Journals are increasingly rejecting papers reporting survey results that do not include appropriate consideration of detectability such as use of Distance Sampling. The increasing capabilities of camera systems is making their use more common and repeated analyses of photographs allows quantification of observer and detection biases and I encourage people conducting surveys to supplement visual observations with digital imagery. I understand Matt's frustration but proper analyses do provide a better quality paper and papers with inappropriate analyses are not cited by researchers that do high qualtiy work.

In my experience in a semi-arid area with highly variable climate, abundance is a poor index. Even if you can count, say, all the plants in a plot, are you measuring the size of each plant? The number of flowers produced? The number of seeds produced? At least attempting to determine the number of viable seeds? Whether they are dispersed? Are you looking at the lifetime of individuals in a population? What's the impact of climate variability? Is the population transitory or permanent? Living things are not distributed randomly--see Legendre P, Fortin M-J (1989) Spatial pattern and ecological analysis. Vegetation 80: 107-138.--so random sampling doesn't work well. In a "transect" of 1.6 million square feet, I certainly can't count all the plants and can't even make what I would consider a valid estimation of annual abundance. I can look back and say that in the past 27 years a particular species is abundant or common or rare and have some confidence in my assessment.

Research has shown that the single most important factor for detectability is abundance. Therefore, to some extent measures of occupancy (or more accurately detection) are related to the abundance.

Yes, indices are full of biases, but so are measures of occupancy. The only difference is that using just occupancy tends to mask the reality. I argue that estimates of detected abundance are more a more honest assessment, but detectability should be considered.

There are biases in all studies - that is why we have the term 'residual error' or 'unexplained variation'. This is ecology!

However...rather than focusing on whether or not detectability is accounted for per se in the published literature, it is the consistency across treatments and the focal question that will determine if using abundance indices is informative or not. Are samples consistent in time and survey method ACROSS 'treatments'? Are the survey methods adequate for detecting for the taxa of interest to the question? If so, then regardless of your detectability biases between species, you can compare treatments and reach informative conclusions. I do agree though that abundance indices derived from observer-based methods have limited use in comparison to trapping/removal-based methods, which are much easier to run simultaneously and to standardise between treatments.

Having said all that, abundance and species richness are usually decoupled in reptiles so raw abundance has very limited inference value in biodiversity assessments between treatments, yet is one of the two community measures reported in almost all papers comparing reptile communities between habitat types.

(response to Melissa) The problem is that 'samples consistent in time and survey methods ACROSS treatments' is not sufficient. Detectability might depend on features of the population (e.g. size distribution of individuals) or the environment (visibility, weather conditions) that change spatially and/or temporally. Hence, detectability will be spatially and temporally variable, which renders abundance indices potentially biased even if our sampling is consistent. Detectability should be accounted for in any case.

Detectability may be related to abundance in some cases, but I don't find that to be the case with high sampling frequency (about once a week) over a long period (27 years) in a rather large "transect" (1.6 million sq. ft.). I define "rare" as no more than one individual per year, and rare plant taxa account for a little over 20% of the 602 taxa of the flora in my study area. Rare species are easier to see than non-rare species, particularly once one has developed a search image for the abundant, very common, common, and uncommon species--simply because rare taxa are usually really different and stand out from other plants. I record alpha diversity of taxa in flower, so it doesn't matter to me whether I see a single flower of a hundred. Not surprisingly, it's much easier to see a flower in anthesis when there are hundreds or thousands blooming, but I sometimes have to look at hundreds of non-flowering plants of a given species to find a single open flower, particularly at the beginning or end of flowering seasons. I do find that the most difficult plants to see are those that aren't there (but have been present in the past), and this takes a really concentrated effort. I learned this when I began to document changes during a severe drought.

Interesting discussion. There is certainly more than just imperfect detection to confound comparisons of abundance indices among studies and over time within studies. But sticking with the detectability bit for the moment, the issue has rapidly come to the forefront - just look at the citation rate of some of Darryl MacKenzie, Mark Kery, and Andy Royle's papers. In my recent experience both submitting and reviewing manuscripts pertaining to occurrence and distribution, I think it has become more or less a required issue to address, especially in the wildlife/vertebrate literature. Increasingly too with plants, which is a good thing. One thing I find rather fascinating though, is the cultural divide among the "occupancy modelers", who now routinely account for imperfect detection, and the "species distribution modelers", who don't. I appreciate some of the recent attempts to call attention to this gap and bridge it - Marc Kery's work in particular.

It is the vertebrate literature i am most aware of and particularly the debate about mesopredator suppression by apex predators in Australia that uses data from track-based surveys of footprints. Apex predators invariably use tracks and trails to move through the landscape (an optimal movement strategy), whereas mesopredators invariably avoid these areas in the presence of larger predators. This is a clear bias that can be accounted for by accounting for detectability differences, but this has not occurred despite publications arising in big impact journals. The outcome is that studies conclude that apex predators (dingoes) limit fox/cat abundance - but using these methods it is clear that detectability issues are driving the published results.

Interesting, sounds like this topic could make for a nice position paper - perhaps a J App Ecol pracitioner's perspective...

A key point in abundance measures is when. Detection of many organisms is highly variable over time (ie. flowers, fruiits, strongly coloured leaves, flyng adults, singing birds, etc). In strongly seasonal systems, repeated sampling may give a better evaluation of abundance.

Counting individuals, measuring their surface or size cannot be applied across the whole range of componets in the comunity: these abundanece estimates only posess a comparative value among similar entities.

Frequencies may give a useful (relative) estimate but results depend on quadrat size and the abundance values depend on individual size and density.

Matt, I agree with your concerns about dubious inferences regarding abundance from indices calculated from different species (although perhaps there is room to use indices to examine behavioural realisations of the MRH). I think it's still possible to use indices to examine changes in animal abundance in some situations, e.g. if variation in detectability can be safely assumed to be negligible relative to expected change in a single population over time. For example, for a pest animal population subject to intensive control, a series of surveys conducted before and after the control program should provide useful information about whether the program had any detectable effect, provided the same monitoring methods and stations are used, and the time scale is short enough to avoid seasonal variation in detectability (and several other important assumptions). The magnitude of effect might be compared over subsequent programs if the conditions are suitable. I don't think it's realistic to directly compare indices across different sites or over long periods at the same site, although maybe ratios of difference can be compared in these cases.

I think the elephant in the room here is the careless use of abundance or occupancy methods just because they seem more reliable or accurate than they actually might be in some cases. These methods are based on many assumptions that are rarely tested, and are sometimes unlikely to hold. Spatially explicit capture-recapture, for example, assumes that animals are independently distributed in space, but how many papers have used SECR to produce abundance estimates for territorial or gregarious species? Occupancy is sometimes used to infer changes in the abundance of pest animal populations where individuals are ubiquitous in the landscape. In this case, even a large change in abundance is unlikely to produce a change in occupancy. I could go on, and on, and on...

Hi Andrew - the pest animal indices are one of the great causes of my concerns. We need to know the behavioural effects of the control and how that affects detectability. For example, it is feasible/likely that removing some individuals alters the movement patterns of the survivors (perhaps by removing territorial restrictions) and hence their detectability. Hopefully they increase their movements and the likely decline following control is underestimated but real. Our data at Scotia using Catling-Allen indices indicate foxes have increased in areas of control and cats are almost absent, whereas camera trapping suggests a completely different story.

Using these indices to compare individual species over short time frames may be valid (as you suggest), but comparing species responses/interactions using them seems fraught given the additional effects on detectability that such interactions cause.

Then there is the problem of insufficient data to run occupancy models, yet people are still comfortable using indices without presenting some measure of error.

For those interested in this topic, I've just posted another question asking for examples of what happens when indices of abundance aren't supported by more robust estimates: https://www.researchgate.net/post/Has_anyone_got_any_examples_of_when_indices_of_abundance_have_led_to_incorrect_conclusions_when_robust_population_estimates_are_later_used#share

Any ideas would be much appreciated.

While I love estimators (especially varaible detection methods) they are far more costly than indices and in conservation often the budget is not there to do those studies as much as we would like to. While indices are a blunt tool sometimes they are the only realistic tool.

A recent paper came out on this topic that I find incredibly entertaining. The authors are very critical of occupancy models and are defensive of indices, which is not so common in the ecological literature these days. Its worth a read, especially the paragraph before the Conclusions section......very funny. I'm looking forward to seeing the MacKenzie-Royle-Nichols group's response to this.

Welsh, A.H., D.B. Lindenmayer, and C.F. Donnelly. 2013. Fitting and interpreting occupancy models. PLOS one 8(1).

Nathan, surely cost effectiveness enters the equation. If the index is cheap but provides useless data, then you've just thrown your money away.

Dan, I've spoken to Daryl MacKenzie about the welsh paper, and he is preparing a response. My understanding of that paper was that it was focused on deriving population estimates from occupancy models, which may be a stretch. That said, some of those authors have published plenty of papers on the assumption that one observation of Species X and 2 of Species Y means Species Y is twice as abundant as species X.

Hey Matt. The paper focuses on how variation in abundance among sites can impart variation into the detection process and bias occupancy estimates. They never evaluate established methods to accounting for these issues such as the Royle-Nichols (2003) model, which is the method I believe you are speaking of that estimates the latent abundance distribution (Although I agree that its a little dodgy). I'm keen to see MacKenzie's response.

I've enjoyed your discussions on this topic and imagine you maybe working up a manuscript on the topic. Looking forward to seeing that, too.

Dan

Making some changes to a Practitioners Perspective for J Applied Ecology now Dan. Fingers crossed they like this version.

Good luck Matt!