The concept of 'correlations between two matrices' of the same dimension can be defined without thinking its meaning. Let A and B be two matrices of type mxn. Since we know from linear algebra that these two matrices can be seen as vectors having dimension m.n, we can define this correlation as XY'/sqrt(Var(X)Var(Y)) where we assume here X is the vector transformed form of A, and Y is the vector transformed form of B, and Y' is the transpose of Y...

what is your purpose of correlation?

What exactly do you mean by 'correlation between two matrices'? Correlation between two variables is understandable, but correlation between two matrices is something unheard of! Can you properly express your problem?

The concept of 'correlations between two matrices' of the same dimension can be defined without thinking its meaning. Let A and B be two matrices of type mxn. Since we know from linear algebra that these two matrices can be seen as vectors having dimension m.n, we can define this correlation as XY'/sqrt(Var(X)Var(Y)) where we assume here X is the vector transformed form of A, and Y is the vector transformed form of B, and Y' is the transpose of Y...

I could not understand. Why should one define something without understanding its meaning? May be what you have defined is correct; but the physical significance should also be understood by the user. Physical significance should be given proper credence, as far as I understand. If I start using some formula without bothering to think about its meaning, that is not proper, I feel.

First of all, we must not forget that every idea begins with intuition and imagination. Then you can impose some meanings or understandings on it. Here, first I have to admit I do not think scientifically. Using only inkling of analogy, for now. If you want to derive a formula, for example, you may take it as (XY')ij/sqrt(Var(X)Var(Y)) where (XY')ij is the (i,j)th element of the covariance matrix between the vectors X and Y. Sincerely...

It is interesting to note that you have made a comment that you do not think 'scientifically'.

Not every idea begins with intuition and imagination. There has to be something that you have observed, and then you can perhaps go forward towards studying that scientifically.

About this matter regarding correlation coefficient, I would like to write some lines. You see, to define a linear space we need to have a Field with reference to two binary operations, say the field of real numbers with respect to the operations of addition and multiplication (R, +, x), and a Commutative Group, say the group of vectors with respect to the operation of vector addition (V, (+)). In addition, the distributive laws with respect to the concerned operations should hold. This gives us a LInear Space V(R)) with reference to the three operations mentioned above.

Now this linear space will be called a Euclidean Space if we introduce the operation of scalar product or dot product, that satisfies four more postulates. The first one is that if u is a vector, then the scalar product (u.u) is greater than or equal to zero.

The vector u can be say (a1, a2, a3). Now Ring theory says that (-1)(-1) = 1. (What I mean is: even this needs the theory of ring to be proved.) As the real numbers form a ring, this product is valid.

This in fact says that the square of a real number is never negative. Accordingly,

(a1)^2 + (a2)^2 + (a3)^2

can not be negative, and hence

the scalar product (u.u) cannot be negative. (That is how mathematics has been growing, following logic, not 'meaninglessly').

Now, if u and v are two vectors, then (u + cv) is also a vector, where c is a real number. So, the scalar product

((u + cv).(u + cv)

is never negative.

Accordingly, we must have

{(u.u) + 2 c (u.v) + (c^2) (v.v)}

greater than or equal to 0.

For this we must have the discriminant

{4 (u.v)^2 - 4 (u.u) (v.v)}

less than or equal to 0.

Or,

(u.v)^2 is less than or equal to {(u.u) (v.v)}.

Hence,

(- sqrt {( u.u)(v.v)} is less than or equal to (u.v), and

(u.v) is less than or equal to ( sqrt {(u.u)(v.v)}.

This leads to the assertion that the expression

(u.v) / ( sqrt {(u.u)(v.v)} )

lies between (-1) and (+1).

Now, say

u = ((a1- a*), (a2 - a*), ....., (a8 - a*)), and

v = ((b1 - b*), (b2 - b*), ....., (b8 - b*)),

where a* and b* are averages of a1, a2, ..., a8, and b1, b2, ..., b8, respectively. (I have taken just 8 values; it can be any finite number of values.)

Observe that from the assertion that

(u.v) / ( sqrt {(u.u)(v.v)} )

lies between (-1) and (+1), we now get

Sum of {(ai - a*)(bi - b*)}/[ sqrt {Sum of (ai - a*)^2} sqrt { Sum of (bi - b*)^2}]

lies between (-1) and (+1).

This simply means in statistical terms that the correlation coefficient of two variables a and b lies between (-1) and (+1)?

Would you now agree that the concept of a Euclidean space is a must before we proceed towards using the correlation coefficient?

That is why I said that I could not understand where from the question of correlation comes up in the case of two matrices! What about the existence of a Euclidean space in this case?

A formula should not be used just because it is there! This is my view; I may be wrong!

Dear Baruah,

I see you have done a very good explanation about correlation coefficient. In fact, all of the explanations you have done are in their places. First of all, I'm a statistician since 1980 and my B Sc is mathematics. I almost completely understood what you told me. But my first answer was approximately 11 hours ago, and my purpose was only to give an answer to the question that it can be defined. I appreciate your effort on this issue, I really care for your effort. But I don't have so much time to discuss on this topic. My interest areas are Markov chains, copulas and hence dependence functions, dependence measures. I want to remind you that: in mathematical disciplines, there are also the so-called problems 'Inverse problems'. You look some of the problems from this perspective at their first stages, whether or not some of their assumptions are satisfied... If I can find some time, I want to give answers to your questions. Greetings.

Thank you!

The first application of correlation between two matrices is in cluster validation in clustering analysis. This correlation is done between proximity matrix and Incidence matrix which is defined as follows:

a. One row and one column for each data point

b. An entry is 1 if the associated pair of points belong to the same cluster

c. An entry is 0 if the associated pair of points belongs to different clusters  

High correlation indicates that points that belong to the same cluster are close to each other.

I hope that shows one application of it.

Supposing I have a 2 dimensional image, and I want to compare it to a "dictionary" of other 2 dimensional images. I can write each image as a matrix, where each matrix element represents the intensity of light in the corresponding piece of the image. To compare the 2 images I would like to know the correlation between the input image and my dictionary images. How would I do this?

The relationship between two matrices is generally different in the horizontal and vertical directions. Therefore two correlation coefficients should be calculated (the horizontal and the vertical correlation) because the data space is two-dimensional and the relationship is generally direction dependent. For calculating correlation between two matrices I developed a new method which is explained in the following paper:

Dikbaş, F. (2017), A novel two-dimensional correlation coefficient for assessing associations in time series data. Int. J. Climatol. doi:10.1002/joc.4998

The study presents a two-dimensional horizontal (row wise) and vertical (column wise) correlation calculation approach where the compared series are considered as two-dimensional matrices in which each row represents a sub-period (e.g. one calendar year of the precipitation data) of the investigated time series data. The method applies a normalization procedure by considering the averages of all rows (namely local averages) for calculating the horizontal correlation and the averages of all columns for calculating the vertical correlation instead of considering the averages of the whole matrices. This enables a separate determination of the degree of relationships between the rows and columns of the compared data matrices by using the horizontal and vertical variance and covariance values that constitute the base of the two-dimensional correlation. I can provide the software if you will not be able to apply the presented methodology.

look at this package in R and the resources in it

https://cran.r-project.org/web/packages/MatrixCorrelation/MatrixCorrelation.pdf

Old relatively technique - Mantel test:

Mantel, N. (1967). A technique of disease clustering and a generalized regression approach. Cancer Research, 27, 209-220.

You may use the "corr2" function of Matlab. Everything is explained in this link:

https://www.mathworks.com/help/images/ref/corr2.html