Hi,
I would like to know if it is possible to validate my clustering algorithm via stability measure? What I mean is, are APN, AD, ADM, and FOM imply on the best clustering algorithm and number of clusters that can be used?
I have done AGNES and Pam for my dataset in r statistical program and my stability optimal result is as follow:
Score Method Clusters
APN 0.07333333 hierarchical 6
AD 0.46335328 hierarchical 6
ADM 0.15485685 hierarchical 6
FOM 0.30368797 hierarchical 3
is it saying that hierarchical clustering is the best method for my dataset?
Thank a lot
- A good work on the stability measure is http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.9082&rep=rep1&type=pdf
- I would also suggest the following is u r using R
https://cran.r-project.org/web/packages/clValid/vignettes/clValid.pdf
Hi Mona,
it is always difficult to answer if it is the best with stability only , for example if you use mixed types of data the stability is certainly not the best (base on the type of transformation you do on the categorical data). Anyway in your case the APN is low, very low and as it not subject to scale your clustering is stable.
The only point in your case will be the intra cluster (FOM) which is optimum for a different number of clusters but as the value is small I would not expect a big change if you go for 6 clusters.
But back to you initial question, you have a good separation between cluster APN, and the element are close to centres, therefore I would conclude that you find a good clustering, is it the best ? Well this depend what you want to do with it as clustering is not targeted toward your final result.
Hope this help.
Alain
I advise you to visualize data by using modern data visualization (data embedding) techniques such as t-SNE (see theano package in Phyton or cafe). You can estimate visually what clustering method is the best>
I would add two comments here:
1. The most stable solution is not automatically the best. Stability is important, but only after solution with good other properties is obtained (e.g. good separation of clusters, ...).
2. If I understand these measures correctly they only deal with stability if certain variables are removed. What about if we would get different sample from the same population? It is also useful to test this, the simplest way beeing (not necessary the best) to split the sample in half, run analysis on both parts and compare results.
- Badges
- Science topic
- Similar topics
- Engineering
- Computer Engineering
- Computer