Dear Dr. Vu,

you have to check whether your data are correlated or not to remove redundant data that can affect the outcome of any clustering algorithm. In addition, normalizing the data based on common criteria can help in speeding up the process and in avoiding local optima.

There are many clustering algorithms that are better than K-means such as FCM or SOM.

Send me a sample to check your data

You can try K-means Clustering Algorithm, DBSCAN, Subspace clustering algorithms or Spectral Clustering among other.

This article might also be helpful - https://arxiv.org/ftp/arxiv/papers/1501/1501.02431.pdf and this article https://machinelearningmastery.com/clustering-algorithms-with-python/

Good luck.

I think that you should definitely preprocess your data before applying any kind of clustering algorithm, in this case. And yes, I think it is convenient for you to preprocess such that the variables of your dataset have mean equal to 0 and standard deviation equal to 1.

With regard to the question about the best clustering algorithm: you did not mention the size of your data, but if the number of observations is not too high I recommend the use of the spectral clustering.

Dear Prof. Dinh Truong Vu,

I investigated your data and I tried the following three steps which works in clustering the data:

1- Normalize the variables with large values base on the maximum value in each column (variable)

2- compute the correlation between these columns and reduce the number of variables (column) according to the correlation value(can be > 0.5)

3- Use Self-Organizing Maps (SOM) with different net sizes and compare the results.

Good luck

You can try true clustering