In order to associate species to functional groups based on a set of traits I want to carry a dendrogram using hierarchical clustering applying the option Unweighed pair group average (UPGMA).
I am using a two-way clustering algorithm, which will cluster both species and traits. Moreover I want to have an idea of the robustness of this clustering. However the softwares I use will build the dendrogram based on a summary data matrix. This means that each column (functional trait) and row (species) will constitute a branch on the dendrogram. Branches are paired in relation to data in the rows for each column and the chosen distance or (dis)similarity.
A bootstrapping option is available to provide a % of times each branch still remains when removing randomly a columns. However this is wrong at least for the analysis I am carrying, because the matrix, is a resulting matrix from a much bigger matrix, which means that removing a column does not have sense, because it summarizes several observations. I assume that each column is ok, and what I want to know is how robust the dendrogram is when randomly removing an observation. The algorithm should remove randomly observations from the data matrix (which is a 1024 samples x nb of species=20), and join (summing up) the data of similar categories (same trait-species) to rebuild the cluster, and then inform about how robust the dendrogram is.
Does such a function exist on a software? Moreover if it does not, and I have to create the function, I wonder what criterion is the most adequate to evaluate the final cluster (for example the % of times each branch is kept as in Pats), or perhaps an information criterion. Then if my set of species is let’s say as big as 60 species, the best of the best will be to have a backward function to eliminate species until building a cluster robust for a expected probability (e.g p
I really don't know if there is an algorithm that does what you are describing. However, there are some methods that can look at the problem as the dendrogram is being built, rather than comparing different "finished" dendrograms. One that I know is the similarity profile (SIMPROF) method. This simply looks for meaningful (significant) divisions in the data using permutations. Before splitting a set of data into 2 branches, SIMPROF shuffles everything around many times to see if having the 2 branches that would be created by the clustering algorithm is significantly different from having branches with totally random members. If not, no further splitting occurs. This is a multivariate, non-parametric approach to assure that branches are meaningful, given that any algorithm will find (meaningless) clusters in a randomly generated dataset. However, I've never seen this applied in 2-way clustering... It is a slightly different philosophy, since it focuses on "meaning" rather than on "robustness" of the outcome, but maybe you can find this useful in some way.
Clarke, K., Somerfield, P., Gorley, R., 2008. Testing of null hypotheses in exploratory community analyses: similarity profiles and biota-environment linkage. Journal of Experimental Marine Biology and Ecology 366, 56–69.
Thanks both Manjari and Miguel. You are right Manjari, that while I mentioned permutation on the title I am rather looking for a resampling technique to confirm the robustness of the branches of the clusters carried.
For more information, the data I am analyzing consist on ants participating on trophic functions. The participation intensity of ant species on functions is deduced from ant species frequency on different baits and their behavior towards them. Even if the number of samples is important (128 replicates of each bait type), the frequency of each ant species on them is not.
The options I generally found to test robustness of clusters are similar to those Miguel has suggested, and are available for one-way clustering. However, as I tried to express on the description, clusters are built from “final matrices” of species x category, and not the whole amount of samples. These methods shuffle columns to test the percentage of times that each branch remains unchanged. But what I want to know is, just as Majari expressed much better than myself, whether the clustering outcome varies or not significantly with random variations on the samples. This is the kind of algorithm I am interested to find.
The SIMPROF subrutin in PRIMEr software: http://www.primer-e.com/primer6_changes.htm
SIMPROF - (similarity profile) routine, which tests for evidence of structure in an a priori unstructured set of samples. In combination with clustering and a new facility to condense specific substructures in dendrograms (by clicking on vertical lines), this can generate trees that are pruned to objectively-defined groups. A factor defining these groups can be automatically created.
Hi all,
Just to point out that should you like to use free software, the command simprof is implemented in R in the package 'clustsig':
http://cran.r-project.org/web/packages/clustsig/index.html
Hope this helps,
Tito
- Badges
- Science method
- Similar topics
- Biological Science
- Ecology