Assuming we have a corpus with unlabeled documents and we have used
a certain unsupervised ML method and we get some clusters.
How can we measure the quality of the built clusters?
Please remember, we don't have the real classification results, other wise we have used a supervised ML method.
One possible way is to select a randomized sample and to manually classify them and then to compare the system's results to these results.
Dear researchers, do you have other ideas how to measure the quality of the built clusters?
Inter-cluster and Intra-cluster distance measures may be useful. Moreover, maximum radius and average radius can also be considered.
You can use a classification algorithm where class label is "cluster".
Or purity - Entropy:
http://www.ijcse.com/docs/IJCSE11-02-03-105.pdf
http://www.ijarcce.com/upload/may/Validation%20of%20Document%20Clustering%20based%20on%20Purity%20and%20Entropy%20measures.pdf
I believe your idea of manually classifying a sample from your data to create ground truth appears the best. You can then apply the standard measures of precision and recall to verify your clustering.
Its a pleasant coincidence that this was question was recently posed. The other day my colleague referred me to: Peter J. Rousseeuw (1987). "Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis".
A brief overview of this concept can be found at: http://en.wikipedia.org/wiki/Silhouette_(clustering)
It seems like this method would provide a solution to your problem.
Let the users decide. In an information system,
I think, users are interested in the results of
a single cluster as result set of one query.
So present the users a HTML page with the
best hits of a variety of clusters. The users
will with a high probability click only on
the results contained in a single cluster.
As far as I know this is what Google is doing:
instead of returning the page with the highest
rank, then the next lower rank and so on,
Google presents some of the highest ranked
pages of one cluster, then some high ranked
of the next cluster and so on.
It is even possible that while you're
clicking on results, Google might deduce
in the background what cluster you're
interested in and switch to results only
from that cluster on subsequent pages.
Assuming users are only interested in
results of one cluster, you can compare
their click behavior with the members
of the most pertinent generated cluster.
Regards,
Joachim
- Badges
- Science topic
- Similar topics
- Publishing