I applied k-means and k-medoids clustering techniques on iris dataset, in particular I clustered with respect to sepal length and sepal width features across 4 classes. With k=3 using k-means I obtained a silhoutte score about 0.42 and then I identified that the cluster labels and class labels are largely different (verified using the crosstab functionality) . With respect to k-medoids, however, with the same k=3, the cluster labels and class labels are almost similar with only a few mismatches. The silhoutte score for this case is only 0.44. Could anyone explain why this is the case?
I understand that silhoutte score provides insights about how data points are centered around the cluster, lesser the value means the data points are sparse and not properly contained within the cluster region. Also for unsupervised setting, we are not supposed to use confusion matrix for estimating the cluster performance. Out of curiosity I tried. I would like to know why silhoutte score does not exactly capture the cluster performance in this context?
Why it doesn't capture the result exactly is complex to explain, and probably more to do with the specificities of the dataset. However, I recommend taking a look at this article on cluster evaluation metrics and why we should look at more than one indicator: https://www.sciencedirect.com/science/article/abs/pii/S0020025521010082
cluster validation is very interesting problem, sil you use is the one without label，so it can not show the quality of final clustering results. By a method without label information evaluating the clustering results is a challenge problem, you can read more papers. The answer is sil has no relationship with final clustering results
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression