I know there are plenty evaluate methods can be used to evaluate the clustering result for a single data set, I am trying to apply the same clustering technique to two different data sets and then compare the similarity of the resulting clusters.
for example I want to compare the result of a same clustering algorithms on two consecutive time intervals with different data.
The answer may vary depending on your goal for comparing the clustering output --- are you trying to examine how the algorithm behaves across known differences in data? Or are you trying to establish structural differences in the data that can be highlighted by the clustering algorithm?
In any case, since the data sets do not contain the same points, the comparison must focus on general characteristics of the clustering output. For instance, if the clustering algorithm is hierarchical, compare the evolution dynamics of the clusters formed: the lifetimes of the clusters, distribution of merging times, growth of cluster sizes. If you can extract the merging times of the clusters, you can use the bottleneck metric to compare differences. I was able to employ this strategy to study clustering behaviour across multiple point samples of a point cloud (see Preprint Intrinsic Hierarchical Clustering Behavior Recovers Higher D...
). Hope this helps! :-)
Every clustering algorithm have varying clustering results. All this is however., based on data preprocessing, size of clusters, and data suitable data-fits algorithms etc.
visit the link below
https://stackoverflow.com/questions/51444384/how-to-evaluate-the-clustering-result-of-two-different-datasets
Paul Samuel Padasas Ignacio : I want to detect change and the extent of change between two consecutive time intervals in order to detect anomaly ( without data access). I have two clustering, how can I compare them and detect change?
Sara Mirzaie: Actually, the link given by Adnan Majeed points you to the same question. So I'm assuming the answers there do not apply to your case. So let me try to understand what exactly you are trying to do. Since you would like to detect anomaly, I would think that while the points in the two sets from consecutive time intervals are not exactly the same points, they somewhat represent the same point cloud and you would like to see whether clustering dynamics change between the two time intervals, and if so, you would like to quantify this change. Furthermore, you do not have access to (or do not want to access) the data so you would try to detect anomaly using only the detected change in clustering. Is this correct?
If so, then this is a bit tricky because there will naturally be changes in the clustering since you are not considering the exact same points. I think what you can do is to first compare the clusters between the two time intervals one pair at a time: take one cluster from the first interval and one from the second interval and use an appropriate metric that compares the two (there are many ways to so this--- euclidean distance, earth mover's distance, bottleneck, Wasserstein). Do this of all possible pairs. Then from all these values you can consider taking some summaries like average, max, min. Once you choose an appropriate single value for a pair of time intervals, you should repeat this process across several more pairs of time intervals to generate a distribution. This will give you a measure not only of the difference in clustering but also how different the clusters are between the two time intervals.
there are no problem to use same clustering algorithm on different datasets if the nature of data are similar. but the goal of this procedure will be to check whether the algorithm may be used upon different data, and compare the results with other algorithms that were used to cluster these datasets. However, if your goal is to use same clustering algorithm upon to different datasets and then compare the results with each other, then the comparison in this case will be un realistic.
Paul Samuel Padasas Ignacio : Thanks for your reply! Indeed, I have a hierarchical architecture. At the first layer I get raw data and cluster them and at the next layer I decide about the change and the extent of the change between two intervals to identify the anomaly. I am monitoring a system with gradual change in behavior, so change is normal. The amount of change is important for me.
Ahmed Albasri highlights a good point. That is precisely the reason why I mentioned in my earlier answer that "while the points in the two sets from consecutive time intervals are not exactly the same points, they somewhat represent the same point cloud". Here, by points I meant data points. Since you are monitoring a time-varying system, the data from the two time intervals describe the same system and you are simply trying to capture the change in the clustering dynamics between the two time intervals. Is this correct? If so, I think your procedure is okay.
Paul Samuel Padasas Ignacio : Yes, I am monitoring a time-varying system and I want to capture the change in the clustering dynamics between the two time intervals. How can I do that?
Sara Mirzaie: I suggested one way in my second answer above. I'd be happy to discuss further via email or a direct message here :-)
- Badges
- Science topic