Dear experts in statistics, I have this question for you. What's the point in performing e.g. hierarchical clustering of a correlation matrix? I do not mean that the clustering is based on the correlations among the original variables, but exactly what I wrote: doing the clustering on the square correlation matrix. so, suppose to have 30 variables each with 100 measures, in the first case you input to the algorithm a 30x100 matrix, in the latter a 30x30 matrix. If we perform the clustering based on correlation (30x100), we get clusters of variables behaving similarly, but working directly on the correlation matrix would give clusters of variables whose pattern of correlation with the other variables is similar. Are the two exchangeable? I would say no, and until now I avoided doing the clustering of the correlation matrix mainly because it was strange to me first to calculate a correlation matrix and then calculate a distance on it to perform the clustering (so "distances" are calculated two times*). However, I am receiving papers to review where they did so and I am not able to understand why they did so and more importantly if there are advantages (except maybe sometimes a nicer figure). Moreover, in principle, if some normalization is performed on the matrix before clustering (for instance standardization on the lines), what once was the same correlation i,j and j,i (i rows and j columns) might become different. In addition, with the correlation matrix you need to perform the clustering of both the columns and the rows simultaneously, otherwise the symmetry will be broken.
Moreover, two variables that are lowly reciprocally correlated might happen to have a similar profile of correlation to another variable. By performing a few examples, it seems to me a very different situation. See figure attached. This is simply the heatmap of log2(C./CC) where C is the correlation matrix of random normally distributed numbers, and CC is the correlation matrix calculated on C (so in fact it is log2 of the ratio of the correlations calculated to perform the clustering on original data, and the correlations calculated on the correlation matrix, as it happens when one use the correlation matrix to make the clustering, without specifying that it is a correlation matrix).
To conclude: is it right to cluster a correlation matrix by calculating a new distance matrix on it? Can someone formalize this from a statistical/mathematical point of view such that there is no way of being contradicted?
*this can be told for instance to the clustergram function, but often in papers, the correlation matrix given as input will be treated as a "normal" matrix of measurements and a new distance will be calculated.
The correlation matrix captures variable interactions in an intrinsically normalised way. If your input data is all on the same scale and the variance for each variable is similar then this step may not make much difference. The main reason for using the correlation matrix is that it simplifies the calculation but inherently eliminating a lot of distracting information and creating a square matrix (which are preferable for many mathematical operations).
In terms of calculating two distances,the first (correlation) is euclidean and projected from a common origin and simply rotates the data into a more easily used form. The second is typically non-euclidean as it is based on distances between multiple points rather than a fixed origin. This means it is worth calculating the two distances.
PCA is a commonly used pre-processing method before clustering and it is entirely based on the correlation matrix, it is a method for unfolding the correlation matrix, with the advantage that you can reduce the amount of noise you introduce into your model by selecting only the PCs that contribute significant explanation to the data. This is preferable to the correlation matrix as it reduces the risk of overfitting the data, so I woudl advise anyone using correlation matrices to reduce the data further using PCA. PCA also addresses your point 'Moreover, two variables that are lowly reciprocally correlated might happen to have a similar profile of correlation to another variable' as this is the scenario that is captured by the various orders of PCs.
I fell the following is a misplaced concern - it is really a question of whether the normalisation if useful or not and will affect both the data and its resulting correlation matrix, so the clustering algorithm will be influenced anyway. To my understanding it is not relevant to the central discussion and more of a tangent.
'Moreover, in principle, if some normalization is performed on the matrix before clustering (for instance standardization on the lines), what once was the same correlation i,j and j,i (i rows and j columns) might become different.
Clustering of a correlation matrix. Available from: https://www.researchgate.net/post/Clustering_of_a_correlation_matrix [accessed Aug 8, 2017].'
Dear James I do not find any answer to my question in your so-long text ;)
The point here is not if calculating the correlations is worth or not. The point here is: when you find that someone made hierarchical clustering of a correlation matrix in a paper (NOT hierarchical clustering of a dataset using correlation as a distance measure but exactly what I wrote in other words use a correlation matrix as input to an algorithm that normally expects the variables), what do you think?
Because if they tell: look, we used this command in R or matlab, and you see that the algo is considering the input a correlation matrix such that it will not re-calculate the correlations/distances among the variables is ok. But if they don't specify the command, and they say we use it as input to the clustergram function in matlab, well clustergram, if you don't add the right command, it will simply calculate the distances over the input matrix (which was a correlation matrix) and perform the clustering on the newly calculated distances. In doing so you will end up clustering not the original variables, but the correlation profiles. And (one of) my question(s) was: will this affect much the output? Given a few tests I did before, the answer would be yes, but can we say something more rigorous? And maybe, I am limited and I don't see any advantage in doing so, but there are some?
why then you speak about PCA?
and the point about normalization I don't think is misplaced. First, I mean normalizing the correlation matrix given as input to a clustering algorithm, not the data before calculating the correlations to be used as input to the clustering algo. If you perform biclustering of a correlation matrix in the right way, you will always end up with the rows and columns in the same order as a consequence of the symmetry of the original correlation matrix. If instead you make what I consider an error, that is you input the correlation matrix to the clustering algorithm without telling it that it should take it as a correlation matrix and not a data matrix, and you perform some normalization, you will break the symmetry and potentially rows and columns will be ordered differently. Now, I think the resulting asymmetry can be considered as an indication of a wrong use of the correlation matrix.
Hello. After a partial read of your question: "What's the point in performing e.g. hierarchical clustering of a correlation matrix?" I would say this is not well posed. A correlation matrix contains statistical expectations of the pariwise variables. It does not make sense to perform clustering on quantities that are already averaged, since this would seem to involve a double expectation step on the same variables.
On the other hand, if the question is applied to sample correlation matrices, whose entries are random variables, the question could be well posed. I believe there are statistics available for covariance matrices that could be applied.Sorry - I can't remember the name at the moment.
Hi Graham, I don't think the question is not well posed. I mean what I wrote. I do not perform such an analysis, I was asking for technical reasons (that you provide, thanks) explaining why you should not do that, since in several papers I reviewed recently, they did it and it was not clear how and if it makes sense. I would conclude from previous comments and yours, that if someone do this *without* telling (if possible) the algorithm that the input matrix is a pre-calculated correlation matrix, and therefore it can use the values as they are after transformation into distances for the clustering step, then it makes no sense to do it (i.e. considering the corr matrix a data matrix on which the algorithm will calculate the correlations). maybe now it's clearer.
Dear Matteo,
I stumbled upon this post of yours from 2017 when I was searching for information on this topic.
Would you have any references in which clustering is performed directly on a correlation matrix? I remember seeing a quoted paper about k-means applied on a correlation matrix, but I can not find it now.
Thank you in advance!
Kind regards
Love
I am also interested in finding an answer for this question. The corrplot package for R has an option to use hierarchical clustering for ordering variables in the correlation plot (with various clustering methods), and you can also draw rectangles for clusters. Isn't this an example of the clustering of correlation matrix? I find this gives a better clustering than using original dataset as an input to hclust function. In my case correlation matrix seems to give a better clustering due to a large number of variables (?).
- Badges
- Science topic
- Similar topics
- Indexes