Dear All,
I want to group my dataset using clustering technique. I apply k-means and used Dunn index for the validation. now I want to know what should be the optimal cluster size based on the Dunn index. For your reference i am uploading the DI graph.
Please suggest what should be the cluster size i need to consider for this plot.
thanks
The maximum of Dunn's index gives the best cluster number(m).
So from the plot, 7 is the best optimal number with maximum of 0.044 in index.
the article attached will explain the principles of different cluster validity measures including Dunn's method.
Hope it will be useful.
Best, Mona
thankyou for your response. The point is that if i keep on increasing the cluster no's the DI value is getting higher after 7. The minimum value in figure is 5. So can't we take 5 as the min no of cluster possible.
From my knowledge,
The principle of the Dunn's index is the maximum of index, the better the cluster number. if we need a lesser number of classes, in case if you say it is 5, it has the minimal index value. So, check where the maximum index lies from cluster number 2-5, Plot shows the maximum at cluster 3, So we can choose 3 instead 5.
If you chose your classes above 5, the maximum is at 7, it's always ideal to take 7. In your case if it's getting higher after 7, you can confine the best as 7.
The best way to rely on is trying different cluster validity methods and check whether it is the same case in all the methods, instead depending on one index.
The above article will guide the better way in that.
I expect better answers from expertise in this field.
Best,
Mona
I use another index Silhouette Coefficient (Figure is shown below ) . Please suggest
Each cluster validity method varies in its evaluation, some methods calculates the intra cluster similarity and some evaluates inter cluster distances or centroids. The purpose of using the right index method depends on our data. In case of Silhouette method, it’s always better to choose Silhouette coefficients nearer to 1, which says it is a good number to choose.
http://stats.stackexchange.com/questions/10540/how-to-interpret-mean-of-silhouette-plot
