Assuming we have a corpus with unlabeled documents and we have used
a certain unsupervised ML method and we get some clusters.
How can we measure the quality of the built clusters?
Please remember, we don't have the real classification results, other wise we have used a supervised ML method.
One possible way is to select a randomized sample and to manually classify them and then to compare the system's results to these results.
Dear researchers, do you have other ideas how to measure the quality of the built clusters?
Inter-cluster and Intra-cluster distance measures may be useful. Moreover, maximum radius and average radius can also be considered.
You can use a classification algorithm where class label is "cluster".
Or purity - Entropy:
http://www.ijcse.com/docs/IJCSE11-02-03-105.pdf
http://www.ijarcce.com/upload/may/Validation%20of%20Document%20Clustering%20based%20on%20Purity%20and%20Entropy%20measures.pdf
I believe your idea of manually classifying a sample from your data to create ground truth appears the best. You can then apply the standard measures of precision and recall to verify your clustering.
Its a pleasant coincidence that this was question was recently posed. The other day my colleague referred me to: Peter J. Rousseeuw (1987). "Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis".
A brief overview of this concept can be found at: http://en.wikipedia.org/wiki/Silhouette_(clustering)
It seems like this method would provide a solution to your problem.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification