Najeeb, you can use both Shannon and simpson indices for measuring alpha (within community diversity). The two indices they provide measures of the different components of diversity. The Shannon index reflects the manner in which abundance is distributed amongst the different species constituting the community. The index is based on the relative frequencies of species in the population, thus taking into account both species richness and evenness. However, the value of the index is most strongly related to species richness. Simpson’s index on the other hand, is a dominance measure since it is weighted towards the abundance of the most common species in a sample rather than providing a measure of species richness. It reflects the probability of any two individuals drawn at random from an infinitely large population belonging to different species, thus the index is less sensitive to species richness.

However, there is no point taking the reciprocal of both. Shannon index has a direct relationship with diversity (i.e. the higher the index, the higher the diversity and vice versa) unlike the simpson index in its original form which shows higher diversity for a lower index. You only need to take the reciprocal of simpson index to allow for a direct relationship with diversity.

Dear Najeeb Iddris,

First of all, it would be good to know what your data structure looks like. What do you actually want to compare?

Regards, thomas

Thanks Thomas..

I have data on trees from cacao farms and from natural forest, with dbh and height measurements for all. I have estimated biomass and carbon stocks already. Now I would like to compare species richness in the different land-sue types, and also compare similarity and diversity. Hopefully that adds some insight. 

-Thanks, Najeeb

Hello Iddris,

Your intent is a bit confusing and not very clear.

As Thomas rightly put it, you need to be more specific in your objectives.For instance are you interested in the importance value,relative density,height,frequency or coverage?Or are you simply interested in the diversity indices of the two landuse types such as  the species diversity of the communities, the degree of evenness, and degree of heterogeneity of the species?

Goodluck, Beckline

Thanks, I am interested in the species diversity of the communities. 

Use both shannon-to show the spread, simpson-dorminance. Also use sorensen index and IVI-important value index

I found two publications that seem to have quite similar research questions as yours (see attached PDFs). Maybe you find some more ideas and approaches there.

Best, thomas

Hi Najeeb , if you are interested in species diversity, I suggest you use either Shannon's diversity  index or Simpson's diversity index but not both. If you use more than one index, then you might be comparing the relative strengths of the indices  (which of course is not your objective). Each of these indices is powerful enough to provide information on the species diversity in a community. You just have to justify why you're choosing one over the many other diversity indices available taking a look at their merits or demerits,  the scale of information the index provides, what  parameters are required & of course your data structure...........

I suggest you read a bit more around the topic first.  There are different ways to express community diversity that suit different questions, and these depend on scale too.  For example are you interested in diversity per unit area (what area?), per number of individuals (what number) or for a total region? 

Personally I think sample diversity and rarefaction (richness per fixed number of individuals) are good initial ways to explore community diversity data as they are both relatively inuitive (many indices are not).

if you dont know it please read the classic Hurlbert paper as a start (there are good reasons it has been cited nearly 4000 times) ! : Hurlbert, S. H. (1971). The nonconcept of species diversity: a critique and alternative parameters. Ecology, 52(4), 577-586.

not available in Researchgate but seems to be online: www.vliz.be/imisdocs/publications/151459.pdf

Good luck ...

Thank you all for the interesting inputs. 

Thomas, thanks for the articles. I seem to have a preference for the study from India, looking at the output of the species richness analysis. Initially I was thinking of using the first-order jacknife to estimate species richness, but i am thinking of perhaps adopting their approach once I can fully understand it. Moreover, our datasets looks the same and the objective at the particular point is similar.

Thanks again!

also read:

Diversity and Evenness: A Unifying Notation and Its Consequences

M. O. Hill

Ecology 54, 1973

Available from Jstor - I think you ca access there at Gottingen?

Mike

The literature on this subject is overwhelming, thanks to everyone for the guidance. This is my take so far from my reading:

1. I am comparing tree species richness between two land-use types sampled with same effort. From my reading, it is possible to use 2nd order jack-knife, chao2 estimator, or Ace estimator to estmate species richness. Observed species richness can be compared by sample-based rarefraction methods.

2. Shannon and simpson reciprocal indices can help describe diversity.

3. I can use either one of Sorensen index, , Bray-curtis index or Morista horn index, in addition to a nonmetric multidimensional scaling for ordination analysis and for comparing community composition.

I will appreciate some feedback if I am wrong in my understanding so far.

Thanks to everyone again!

Hi Najeeb,  what is your justification for choosing the reciprocal indices of Shannon & Simpson for estimating diversity ?

Najeeb, you can use both Shannon and simpson indices for measuring alpha (within community diversity). The two indices they provide measures of the different components of diversity. The Shannon index reflects the manner in which abundance is distributed amongst the different species constituting the community. The index is based on the relative frequencies of species in the population, thus taking into account both species richness and evenness. However, the value of the index is most strongly related to species richness. Simpson’s index on the other hand, is a dominance measure since it is weighted towards the abundance of the most common species in a sample rather than providing a measure of species richness. It reflects the probability of any two individuals drawn at random from an infinitely large population belonging to different species, thus the index is less sensitive to species richness.

However, there is no point taking the reciprocal of both. Shannon index has a direct relationship with diversity (i.e. the higher the index, the higher the diversity and vice versa) unlike the simpson index in its original form which shows higher diversity for a lower index. You only need to take the reciprocal of simpson index to allow for a direct relationship with diversity.

I found the Simpson diversity index suitable for me. Simpson’s index on the other hand, is a dominance measure since it is weighted towards the abundance of the most common species in a sample rather than providing a measure of species richness.

Thanks lot good questions and suggestions from all. you can use both Shannon index and Simpson's index for the calculation of tree species diversity even you can do chi-square test for the two sites.

Najeeb,

Douglas's suggestions provide very good advice on how to proceed. Explore your data, do a Whitakker (log abundance vs species rank) plot to estimate the distribution type, and rarefact to a fixed size or estimate total S through Chao's indices.

However, from your description I'm not sure  whether you are comparing diversity from two samples taken at two sites, or from many samples taking at these sites. Community distance metrics (Sörensen, B-C, M-H, etc.) are not really relevant for a single comparison between two samples if that's all there is to it. They would however serve to establish the validity of the samples as well as the difference among between sites in a multi-sample study, as samples from a site should be in a cluster and samples from the other site should cluster separately.  If the clusters aren't clear then the difference between sites will be unclear too.

Comparing richness or diversity between sample pairs can be made provided that sampling effort/size are equal (as seems to be the case) and samples meet minimal sample requirement, i.e. it is reasonable to assume that the community structure has really been captured.  A species accumulation curve is an initial approach to trying to establish whether samples are amenable to direct comparison. But a statistical comparison will require you to bootstrap your inventories to get a pseudovariance for the richness. As for Shannon's H', you may use Hutcheson's (1970) estimates of variances and DF to compare H' values of two samples. Zar (1974: Biostatistical Analysis) will give you the formulas. (I can send you an Excel file to do the calculations, too).

Here are a couple of links on the problem of minimal sampling and data sufficiency, one of them for African palm trees. Regards

Article Knowledge of diversity of wild palms (Arecaceae) in the Repu...

Chapter Optimal Sampling for Complexity in Soil Ecosystems

more than enough for the first above indices. Apart from this, u may calculate

IVI, Complexity index, Niche width, MIV, 

All the best!

Hi Najeeb, I found all the previous comments really constructive and informative about the use of Shannon and Simpson indices, however I can suggest a couple of things. First, I think you can use both indices in order to make your study more comparable with other works that only use one of them, it is just important to clarify why are you using each one, though Simpson may be more useful. Second, because these indices are just numbers whitout a significance test, it is imposible most of the time to say when a community is more diverse than any other (e.g. H1=0.64 vs H2=0.81), therefore you can try either the procedure for testing for differences between two diversity indices described in Zar´s book Biostatistical analysis or any other bootstrap-based resampling technique. Third, and more important, you have to take into acount that these indices have no a biological meaning about the species identities or the community you are studying, therefore you may want to show diversity in terms of functional groups, lifeforms, native vs endemic vs exotic spp, specialist vs generalists, phylogenetic relationships, etc.

I also want to suggest you to try to compare richness by individual-based rarefaction methods, which gives, most of the time, a different measure compared to sample-based rarefaction when individual densities differ between habitats. For more detail see Gotelli & Colwell 2001-Ecology Letters  and  Colwell et al. 2012 in J. Plant Ecology.

On the other hand, if you want to perform an analysis of similarity and to compare communities composition it can be useful to apply an analysis of similarity ANOSIM, which provides a measure of statistical significance, then you can try with the SIMPER analysis to see which species are the responsible of the differences, if that is the case. All these analysis can be performed with the software PAST, including Sorensen, Morisita and dendrograms.

Good luck, with your work.

Thank everyone of you for such constructive inputs. I have found most of the literature extremely helpful and I seem to understand what I am doing more and more. 

Thank you Mike, I was able to access the article and it's clarified a lot of questions I had. 

Arturo, thank you very much. Can you kindly send me the excel? I will be much grateful.

-Najeeb

Deer Najeeb

It seems very interesting and constructive discussion and you may already have your feedback. Regards

Hi Najeeb,

I think that especially Uzoma and Arturo target your question. I would also recommend to approach the plant communities in your research by looking at them always as a whole of interactions and dynamics. Best interpretations of plant communities behavior(s) can normally be reached by using the different indices as 'integrated' components of the same reality. Microclimate and microsite analysis can also help.

Hi Najeeb, this is a simple excel originally by Rebeca Jann (Queens University, Charlotte) with some further enhancements. It will allow you to calculate significance for differences in H' (as measured in natbits) through the Hutcheson test. There are a few instructions within but please refer to the Zar book. The Krebs manual (Charles J. Krebs, 1998: Ecological Methodology, 2nd edition, Benjamin Cummings) also has a description of the method. Regards

Thank you Arturo for the excel file and inputs. So to follow up on my question, now my reading has brought me down to these understandings and questions:

First, for clarification towards an earlier question you asked, I am comparing two land-use types (forest and cacao agroforest) of which multiple samples (12 replicate plots each) have being collected on trees (dbh and height). I have decided to go with sample based rarefraction to control for uneven abundance between the land-use types, and then compute rarified curves using Mao Tao estimator for comparison of species density (not richness) since site conservation is my objective. 

Then I plan to estimate tree species richness with Chao 2 and Jacknife 2 estimators which seems to be more robust from papers I've read. Then I will use the exponential of Shannon entropy and Simpson's reciprocal index to explore diversity between the land-use types.

Still undecided about how to explore community similarity. I've read about the pros and cons of most of them, and even though Bray-Curtis seems widely used, some studies advice against it. Morista Horn index also seems to have its disadvantages, which makes me lean towards the Chao-Jaccard similarity index. Am I in the right direction, given that my  data has species that are very abundant than other?

What about using nmds ordination to visualize community composition? is it the same as the similarity index?

Sorry for such lengthy text, and I appreciate the time taken to guide me. Thank you everyone.

Best, Najeeb

Najeeb, your analytical plan seems sound as regards to richness.--but that's something you should better double check with your Masters advisor(s).  As for the similarities, much depends on the nature of your data, and therefore your advisor(s) should also be best positioned to offer significant advice other than the generalities we are covering here (I mean, without a look at the actual data it is harder to be very precise).

That said, you seem to be concerned about how overly abundant species in some samples play out, which, as you correctly predict, may overkill when using metric indices. To that I can only say that if all premises for comparisons are met (you are dealing with equivalent species, e.g. you aren't comparing herbs on one side with trees on the other), having abundances or coverages adds information to the comparison and therefore distance- or information-based metrics such as B-C or M-H are perfectly valid, in my opinion. If you prefer to use multivariate comparisons, then factor analysis may help you observe the big picture.

The decision to forego abundances and stick to presence/absence only is linked to stressing the species composition importance, rather than the species fitness importance. That in turn is an incomplete answer to the question of roughly whether species could reach there vs species could thrive there. So much depends on exactly what question are you trying to address. If it is what species are there (e.g. presence of indicator species) what is really important to your question, then binary indices are certainly best suited, and in the multivariate arena then ndms (roughly a "rank-only" factor analysis) could be very adequate to explore your data and discover potential groups, possibly to be later confirmed through formal tests. Yes, ndms can help you visualize your community composition but although it should highlight groups of samples, it does not establish a hierarchy as a similarity-based cluster would. (Let's say that ndms will help you delimit groups allowing for internal fuzziness, while a cluster will provide you a hierarchy out of several possible ones.)

An additional question you may need to ask is whether absences of species are important. The original Jaccard and its derivative binary indices stress common species (those appearing in the two samples being compared) but you may want to group samples that lack similar sets of absent species, for example to highlight some environmental factor that affects a group of samples by banning certain indicator species. If such double absences are important as indicative of something in your samples (think e.g. a soil parameter that can be off-mark such as acidity), then it's better to use a similarity index that can look at those absences, such as simple matching coefficient or its many derivatives.

You will also need to decide on which cluster algorithm you want to use. Most literature tends to resort to UPGMA clustering but, again, your particular dataset, sampling area, conditions, and questions to ask may suggest otherwise (e.g. simple or complete linkage if you want to, respectively, stress what glues or what separates sample groups). Again, you should ask your advisor or possibly a statistician familiar with your project.

Again, the literature offers a whole host of case studies that may be comparable to yours. Just a little warning--being most frequent in literature does not mean that a particular method is "better" overall. First, simple indices e.g. binary have had a much longer run on account of their easier calculation. Then, on the metrics side, Euclidean distances have been used for such a long time that there is a large corpus of literature on that. Newer indices may have been used less but that does not mean they are less useful. Understanding how an index, metric, or test works, and what are its limitations, is key and paramount to a successful application.

Sometimes, "playing" with the data (testing different methods in light of the known behavior of such methods) offers insights in the structure of your data. Benchmark-type packages such as PAST help you do that with relative ease, although you'll probably find a wider dictionary if you speak R.

Sorry for the protracted answer. It is a complex and open issue, really, as surely you've seen in the literature. Work with your advisor, project's colleagues, and field people (those that actually collected the data, if it wasn't you) to find the path best suited to your particular dataset and objectives. Regards and good luck.

1. Jarvis, Devra I, Anthony HD Brown, Pham Hong Cuong, Luis Collado-Panduro, Luis Latourniere-Moreno, Sanjay Gyawali, Tesema Tanto, Mahamadou Sawadogo, Istvan Mar, Mohammed Sadiki, Nguyen Thi Ngoc Hue, Luis Arias-Reyes, Didier Balma, Jwala Bajracharya, Fernando Castillo, Deepak Rijal, Loubna Belqadi, Ram Rana, Seddik Saidi, Jeremy Ouedraogo, Roger Zangre, Rhrib Oum Keltoum, Jose Luis Chavez, Daniel Schoen, Bhuwon Sthapit, Paola de Santis, Carlo Fadda and Toby Hodgkin 2008.A global perspective of the richness and evenness of traditional crop genetic diversity maintained by farming communities. "Proceedings of the National Academy of Sciences PNAS (USA):1-6; www.pnas.org/cgi/doi/10.1073/pnas.0800607105

Hi. Shannon and Simpson diversity are special cases in a continuum of diversity measures. Maybe start with "Generalization...". All indices and partitioning are implemented in Entropart. Best,

Article Generalization of the Partitioning of Shannon Diversity

Article entropart: An R Package to Measure and Partition Diversity

Article Practical Estimation of Diversity from Abundance Data