Hello,
I am working with an invertebrate data set (i.e., counts of individuals per invertebrate order, captured by pitfall trap) and am exploring trends in community composition in relation to environmental attributes. In particular, I am exploring if there are differences in community composition (as captured through pitfall traps) between two neighboring islands by visually exploring trends via NMDS (with wisconsin standardization, using Bray-curtis dissimilarity) as well as post-hoc/resemblance-based permutation methods. My experimental design, though, is unbalanced, as I have 329 samples from island 1 and 121 samples from island 2.
First, I used a PERMANOVA to detect differences in the locations (centroids) of my two groups (island 1 and island 2). When using the PERMANOVA test, it specifically tests the null hypothesis: "the centroids of the groups, as defined in the space of the chosen resemblance measure are equal for all groups." So, as explained in Anderson and Walsh (2013), if one were to fail to reject the null hypothesis, then any observed differences between the centroids in the data set would be similar in size to what would be obtained under random allocation of individual sample units to the groups. When I used the PERMANOVA test on my data set, I had the following result (F = 37.826, R2 = 0.07786, and p < 0.001).
However, Anderson (2001, Fig. 4) warns that the method may confound location and dispersion effects: significant differences may be caused by different within-group variation (dispersion) instead of different mean values of the groups. PERMDISP is a common test completed in conjunction with PERMANOVA and tests the null hypothesis of "no difference in dispersion between groups." This test can be quite helpful, as it can identify if it is the dispersion of the group data from the centroids that is driving the significance (of the PERMANOVA test) or if it is the centroids of the group data themselves. However, for both the PERMANOVA and PERMDISP tests, it is ideal to have equal sample sizes to include this; unbalanced experimental designs can either increase rejection rates or the test can become more conservative.
In any case, I ran the PERMDISP test on my data and received this result: F = 48.346 and p < 0.001. Given that I have unequal sample sizes between my groups, I cannot conclude if it is dispersion alone or both dispersion and centroid differences that are driving/affecting the result of the PERMANOVA test.
So, my question is--with an unequal experimental design (something that I can't change), what the best steps for trying to interpret trends in this kind of ecological data, given the significant test results from both PERMANOVA and PERMDISP? What are the next logical steps from here?
Either the centroid and/or the spread of the objects is different between the groups on PERMANOVA. You could use principle component analysis (PCA) to explain the variance-covariance structure of your variables through linear combinations. Or you could use the Kruskal–Wallis test, which is a non-parametric method for testing whether samples originate from the same distribution. It is used for comparing two or more independent samples of equal or different sample sizes. Don't be limited by the PERMANOVA if it takes you to a dead-end :)
Thanks for your feedback! That was the same conclusion that I came to as well--that either the centroids of the groups are different or the dispersion of the two groups is different. I'm planning on exploring PCA instead and examining the eigenvectors and eigenvalues. I'll definitely check out the Kruskal-Wallis test, too. I hadn't thought of that and was going to look into the Hotelling's T-square as an alternative.
Hi fellow ecologist,
If you want to make sure your effect is related to position and not dispersion, you can test (via vegan::betadisper, =PERMDISP2) whether dispersion is similar within the two islands. This would be analogous to bartlett.test in linear regression.
https://www.rdocumentation.org/packages/vegan/versions/2.4-2/topics/betadisper
Moreover, with simply two islands, no need for any post hoc testing, if there is an effect than island 1 is significantly different from island 2.
The unbalanced nature of the design is problematic only in the case of small sample sizes (see notes in the r function betadisper), which I would not say is the case here.
Hope this helps,
GA2
Thanks Guillaume! I used the PERMDISP test as well with my data set, which did work; my question was more about how to interpret results using the PERMANOVA and PERMDISP test when sample sizes are very unequal. When I did my PERMANOVA test between islands, the two islands showed an extremely low (but significant) difference. Although the difference is quite weak, it likely indicates that there are different orders present (or present in different abundances) within each tested island group. In addition, the R2 values are very small for the factor (island) used; most of the variation in the data (more than 70%) is NOT explained by the factor. So, that means that I am missing environmental data that could better explain the variance in the invertebrate community data; however, unfortunately, the PERMANOVA test (via adonis) doesn’t cope with nested designs or multi-factor designs (particularly for unbalanced data) very well, as it does not construct the F-ratios and p-value permutation algorithms by reference to expectations of mean squares (EMS), which is essential for nested/hierarchical ANOVA-type models and designs (per M. Anderson). For islands, PERMDISP was statistically significant, so I have evidence that there are significant differences in the multivariate dispersions (spread, or variability in community structure) of invertebrate assemblages for the two islands. More specifically, there is greater variation in community structure for island 1 (where mice are present), than for island 2 (where there are no mice).
Welcome to the club. I can only agree with everything you said. Most people tend to overlook these nested structures, especially when complex, thereby producing meaningless p-values. However, first order nesting (e.g. blocks in an RCBD) can be specified via the "strata" command and more sophisticated designs can be specified through the "permutation" argument in adonis2. Gavin Simpson proposes a few - relatively simple - examples of restricted permutations in:
https://github.com/gavinsimpson/esa-advanced-vegan-2015/blob/master/advanced-vegan-slides.Rmd
His examples are based on CCA but it's the same thing (and package) for adonis2.
The last time I dealt with complex structures (temporal and spatial blocks, pseudoreplication), I simply specified that correct permutations could not be specified and that only % of explained variance would be retained (as this is not influenced by permutation design).
Again, I think unbalanced data is only problematic in case of low sample size, as specified in the note of the function.
Finally, you could image sampling n times 121 points from the 329 and see how this influences the results.
Best luck to you,
GA2
Thanks for passing along the link; I'll definitely check that out! That sounds like a really interesting avenue to explore, especially given the fact that currently I can only do a one-way test with adonis for PERMANOVA. Thanks again!
- Badges
- Science method