Dear Lucy,

Clustering accuracy can be measured by Silhouette index. To improve index, discard the outliers (if any) present in the data.

Dear Lucy,

Clustering accuracy can be measured by Silhouette index. To improve index, discard the outliers (if any) present in the data.

Hi Lucy,

You may also refer to the following paper:

Sherkat, E., Nourashrafeddin, S., Milios, E. E., & Minghim, R. (2018, March). Interactive Document Clustering Revisited: A Visual Analytics Approach. In 23rd International Conference on Intelligent User Interfaces (pp. 281-292). ACM.

Dear Lucy,

In order to find out the outliers present in the dataset, you may use DBSCAN clustering algorithm that requires Epsilon and Minimum Points as parameters for threshold values. The better selection of Epsilon and MinPoints values can be obtained using k-dist graph.

The Silhouette index value simply expresses how good clusters are formed (similar points are placed together in a cluster).

Dear @Nadeem,

Thanks for your reply.

What is k-distance graph? how it can be used to get the Eps and Min points?

Dear @Sobhan Sarkar

Yes, but how can I decide which points are the outliers?

What is the threshold? (when to decide that a distance of a certain point from the center is large (above threshold) so it should be considered as an outlier...)

Dear Lucy Anderson , dear Sobhan Sarkar -

I am interested in application of Silhouette coefficient for data containing some outliers, i.e., singleton-clusters. The point is that the Silhouette coefficient of singleton-clusters is zero, which means no strong structure is found. However, every outlier should be a singleton cluster and needs to be assessed as a good structure, in my opinion.

@Lucy: Could you solve your issue with Silhouette coefficient?