Instead of Shannon, I would suggest using a richness index (such as Menhinick or even pure species richness) in combination with an evenness index (e.g. Pielou's J). These offer a much more insightful picture of community structure.

I would consider using methods such as the Chao 2 estimator of species richness if you work with accumulative species lists, such as those of Mackinnon (useful for birds, butterflies, fishes, etc.). Two interesting references:

MacKinnon S, Phillips K; 1993. A field guide to the birds of Borneo, Sumatra, Java and Bali. Oxford Univ. Press, Oxford.

O’Dea et al. 2004. Rapid assessment in conservation research: a critique of avifaunal assessment techniques illustrated by Ecuadorian and Madagascan case study data. Diversity and Distributions 10, 55–63.

Hi all,

Species richness, Shannon entropy, Simpson diversity are all diversity measures that give different weight to the contribution of relative abundances to diversity. So, these diversity measures are in fact a continuum from species richness (relative abundances have no role in the index) to diversities that depend only on the most abundant species (Simpson and others...).

I would recommend Jost 2007 (Ecology 88:2427–2439. http://dx.doi.org/10.1890/06-1736.1) for a much better explanation.

Hope this helps.

Andrés

Among standard diversity measures there are really only 2: richness (the number of things) and richness (the relative abundance of things). All standard measures are one or the other, or a blend of the two which varies according to context. This can clearly be shown using a multivariate analysis of the behaviour of indices across samples. Which type of measure is 'best' depends entirely on the details of the scientific question being asked. They are not alternatives, they are different things.

An alternative approach is to look at a sample-size independent measure, such as the average relatedness among individuals or species through some hierarchy or distance matrix, such as taxonomy or function. See papers by KR Clarke and RM Warwick on taxonomic distinctness for examples.

Thanks all for you input so far on this simple yet important question.

To provide a bit more backstory as we keep this conversation going, I am presently in the process of writing a manuscript on a fish community diversity dataset that I have analyzed. And recently I spoke with a colleague who recommended that I remove Shannon's index from the paper. So, this led me to question which index is best etc. And as I expected, you are all essentially giving me different answers as to what you think the best diversity metric is!! That said, to Andres, I now realize that I should have said that I know very well what the most common diversity metrics (richness, Shannon's, Simpson's) are and the differences between them, but I appreciate comments on this point too. And I am going to read the Jost resource. I have read some of his papers, but this might contain information that is new to me. Still, Andres, could you comment on which diversity measure you think is best for publication?

I could use better Rényi diversity in case of lacewing assemblages than Shannon or modified Shannon-Wiener diversity indices.

http://digitalassets.lib.berkeley.edu/math/ucb/text/math_s4_v1_article-27.pdf

The dispute about the "best" index cannot be answered finally and it will probably never end. My personal decision is to prefer common metrics for simplicity, and to employ multivariate statistics for more insight. -- Thomas

Hi again,

In my opinion there is no a "best" index, because each index measures different things... Depending on you aims, you have to choose one or the other. Do you give a lot of importance to rare species, then go for species richness. Do you prefer to not give too much importance to the contribution of rare species to diversity, then go for the exponential of shannon index, and so on...

Cheers

Andres

should apply Shannon's index for areas such as estuaries and the sea. These areas are characterized ecosystems such as streams, rivers, only use Shannon's index as a reference.

Shannon wiener index has been reported to be more accurate than simpson index. i am surprised your colleague asked u to remove it from the paper.

Hi,

For comparing diversity among different sites you could calculate four indices of the Hill’s diversity series(Hill, M. O., 1973. Diversity and evenness: a unifying notation

and its consequences. Ecology 54: 427–432.)

N_alpha=(sum:p_i^alpha)^1/(1-alpha)

(note It is like Rényi diversity index series)

If you calculte this using alpha values 0, 1, 2, inf. then you get

1. N_0: the number of species;

2. N_1: the antilog of the Shannon index

3. N_2 :, the reciprocal of the Simpson dominance index, 1/D

4. N_inf: the reciprocal of the proportion of the most abundant species, Berger–Parker (BP) index

The indices take increasingly into account the ‘‘dominance’’, the importance of dominant species increases with increasing alpha therefore you could get much more explanation than if you use only one diversity index.

This example can be found in: J. Leps: DOI 10.1007/s10750-013-1723-2

Hi all

I really stand up for Jost's (2006. 2007) approach that was recently reviewed by Tuomisto (2010a, b,c, 2011). They based their theoretical approach on Hill's foundation work to tie up all the metrics together and solve some of the most counter-intuitive aspects of traditional diversity index.

I really recommend them

F.

Tuomisto's paper may be a bit technical to read, I suggest to start with

Tuomisto, H. 2010. A consistent terminology for quantifying species diversity? Yes, it does exist. Oecologia 164: 853-860.

and

Tuomisto, H. 2011. Commentary: do we have a consistent terminology for species diversity? Yes, if we choose to use it. Oecologia 167: 903-911.

and deal with the others only if interested:

Tuomisto, H. 2010. A diversity of beta diversities: straightening up a concept gone awry. Part 1. Defining beta diversity as a function of alpha and gamma diversity. Ecography 33: 2-22.

Tuomisto, H. 2010. A diversity of beta diversities: straightening up a concept gone awry. Part 2. Quantifying beta diversity and related phenomena. Ecography 33: 23-45.

Thanks all for sharing your thoughts, and especially for sharing citations and papers with me! I have enjoyed reading through some of this material, and when I have more time and finish the readings I hope to come back and synthesize what I've learned here. However, the general pattern that appears to be surfacing thus far in our conversation is one of two camps, those (1) comfortable with Shannon's index and modifications of this index (or other traditional ones), and (2) those who favor indices related to Renyi and Hill numbers, including metrics suggested in Jost's work. One question I have about this is that I am aware of how to calculate several of these metrics, including software for computing Shannon's etc., but what software is good for calculating Hill's numbers and similar approaches? Does this need to be done by hand / self-programmed?

You could use the PRIMER software (Plymouth Routines In Multivariate Ecological Research) to calculate the diversity index.

PAST folk.uio.no/ohammer/past/‎ easy to use and could calculate a lot indexes or R

Dear Justin

I developed an R script for calculating True diversity, as well as for decomposing it into its alpha beta and gamma components. The script is embedded in a manuscript currently under review. As soon as I receive an (hopefully positive) answer on my manuscript I'll be happy to have it shared

F.

Is this question really relevant? We have been discussing for at least two generations of scientists on that. Once you got a measure of "biodiversity" as you call it, what are you going to do? Can you effectively use it no matter the index you adopt and for what?

Surely any index you use is context dependent i.e. it depends on your question, your study system, habitat type etc. In any case, reducing complex dynamics into a single (or a few) metric(s) will always create problems, but it is of course sometimes necessary, especially in comparative contexts.

Species richness, the number of different species in an ecosystem, was for a long time the predominant measure of global biodiversity. This is now changing rapidly (see a recently published paper in Nature) as researchers are realizing that rather than considering the sheer number of species, it is which specific species that are present that is crucial for ecological functions underpinning the provision of ecosystem services.

Dear Giovanni,

Would you be as kind as to show the accurate URL of this article?

It is probably slightly off-topic, but I have written a library of functions for use in ecology:

https://www.researchgate.net/publication/235945467_Library_of_functions_for_(paleo)ecology

Data Library of functions for (paleo)ecology.

Dear Andràs, here is the URL for the article:

http://www.nature.com/nature/journal/v501/n7468/full/nature12529.html.

I would recommend to read it carefully.

Best everyone

I agree with Giovanni. With the rise of easily accessible phylogenetic trees and functional trait databases, just analyzing species richness, in whatever form it takes (PIE, Shannons, Simpsons, S, and etc.) seems rather outdated. Rapidly, it seems, functional diversity and phylogenetic diversity, and their associated metrics (e.g. Rao's Q, FDis, and etc.), are now being used to answer more specific questions about ecosystem function, community assembly, and conservation priorities (though, I may be biased because I am co-author on a couple of papers in submission on the subject). Also, I agree with those that bring up Jost's work on using numbers equivalents in dealing with the "doubling property" issue. This issue is also important when analyzing phylogenetic and functional diversity metrics as well (see Chao et al. 2010. Phylogenetic diversity measures based on Hill numbers. and Chiu et al. 2013. Phylogenetic beta diversity, similarity, and differentiation measures based on Hill numbers).

To directly answer your question: it really depends on your question. If you insist on just quantifying taxonomic richness, how important is evenness in answering your question? Each metric has its pros and cons - it is just a matter of selecting the right tool to answer your question.

Hi Justin!

I tend to include both a richness measure and an evenness measure when exploring environmental effects on communities. A very strong effect may have consequences for the number of organisms present, whereas effects that are weaker yet still important may change the relative contributions of species to the community without changing the numbers of organisms. I would bet many of the measures will covary as well, which could be a good post-hoc guide to what measures provide unique information. Let me know what you find!

I agree with the points that Aiden Vey made in his post. The appropriateness of any index that you use depends on what you are using it for. All too often, one sees researchers reducing datasets rich in ecologically information to a single number thereby losing that information. Using the data used to calculate Shannon's diversity index (relative abundance of different taxa) can reveal a great deal about type of habitat present, insights as to how different communities differ, which species are playing which ecological roles in those communities, what dynamic is happening and why.

If a metric is the best way to investigate the research question that you are trying to answer then use a metric that it sensitive to changes in the independent variables that you are using. If a metric is not the best way, then use the relevant ecological information to investigate the research question. For example, many researchers use species richness, Shannon's species diversity index to measure the impacts of pollution on macrobenthic communities. The problem that is encountered with the use of indices in this case is that they are not sensitive to medium and lower levels of pollution that have a significant effect on the community. The relative abundance or biomass of higher level taxa, e.g. annelids, molluscs and crustaceans is far more sensitive to pollution than species richness or species diversity. Their relative abundance and changes in them over time give profound insights as to the level and the type of pollution impacting on the community.

So as you develop your research plan, consider what are the most suitable variables to use to answer your research question. What conclusions can you draw from a metric and what conclusions can you draw from the data you used to calculate the metric? then make your decision.

Dear Justin,

you can find the R script for calculating True diversity and decomposing it into its alpha, beta and gamma components in our last paper:

Ground Layer Plant Species Turnover and Beta Diversity in Southern-European Old-Growth Forests

Francesco Maria Sabatini, Sabina Burrascano, Hanna Tuomisto, Carlo Blasi

PlosOne 2014 - http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0095244#s6

This is the direct link to Appendix 1:

http://www.plosone.org/article/fetchSingleRepresentation.action?uri=info:doi/10.1371/journal.pone.0095244.s002

I hope this helps

F.

Nulls models in ecology. Nicholas Gotelli

Dear Francesco, but once you get  the "True" diversity and decompose it into its alpha, beta and gamma components what are you going to do with it?  I am really curious to know.

There are several different diversity indices. You've already mentioned the Shannon-Wiener and Simpsons indices, but for clarity's sake I'll explain their purposes. The Shannon-Wiener Index uses uncertainty to measure diversity, while the Simpsons index calculates for diversity as a function of dominance. Consequently, the first is a better measure of species richness, while the second is a better measure of relative abundance or evenness. This, I find, is an apt illustration of how there is no "best" method to measuring diversity - choosing an index really depends on the purpose of your study. It maybe helpful to ask yourself: "what do I want to find out about this community's diversity?", and then selecting the appropriate index.

To get you started, here are few other indices for you to look up: Berger-Parker, Brillouin, Alpha Diversity, Beta Diversity, and Gamma Diversity.

Hope this helped!

The concept of "diversity" is ill-posed, IMHO. Matthew is right. Everytime, first ask "what I do want to know". That's all. I've programmed a bunch of diversity measures, but I'm now not sure it was worth the time I've spent. https://github.com/ThePaw/go-eco/tree/master/eco/div

P

I wonder if the same analysis can be done for the measurement of diversity for a population of pathogens.....