I am checking different methods to perform hierarchical clustering with a very large number of elements (between 100,000 and 500,000). I have tried many different software like MeV, Cluster, etc. even using computer with a huge amount of ram it is impossible to visualize the tree. I would also visualize the output tree in order to select some clusters.
Do you know if this procedure can be performed?
Thank you
You cannot see all the elements in one graph although the clustering is finished. I think you should divide your data into subgroups before clustering if possible.
I've had good experience with CLUTO previously http://glaros.dtc.umn.edu/gkhome/views/cluto for clustering (not visualization). Now I see they have a visualization module gCLUTO in their software, maybe you can give it a try, I haven't tried it myself yet.
As for visualizing large hierarchies, these may give you some ideas:
http://bioinformatics.oxfordjournals.org/content/25/4/557.full
http://web.cs.ucdavis.edu/~ma/papers/graph2.pdf
http://bioinformatics.psb.ugent.be/hypergeny/index.php
http://www.randelshofer.ch/treeviz/
There are some good links to literature in the earlier posts.
There are some issues you should consider, before clustering. If your matrix is R X C where R is the number of rows (about 100,000 to 500,000 in your case) and C columns. Then each row is a vector with C components, and clustering using HCA and eucledian distances do not make much sense if you have more than 10 or so columns.
You can take a random sample without replacement from the matrix and then cluster it, or you can try dimension reduction first with Principal Component Analysis and at least reduce the number of columns for the matrix.
Just a comment on the pre-assumption that a larger number of features should make a more fragmented clusters -- it affects the number of LEVELS in the tree hierarchy, not the aggregation to a tree itself.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Bioinformatics
- Bioinformatic Tools