Full rank does not necessarily mean that rows or columns of a matrix are not highly correlated.

For example:

A = [ 1, 2, 3

1, 1.9, 3.1

0.9, 2.1, 2.9 ]

Using the online tool below we see that A has full rank.

https://onlinemschool.com/math/assistance/matrix/rank/

But the rows of A are highly correlated. For example, using the online tool below, row 1 and row 2 have correlation coefficient 0.997

https://www.mathsisfun.com/data/correlation-calculator.html

Respected Dr.Thomas W Kelsey,

Thank you very much for answering the question, this is where I get confused.

We have a definition for random variables, that if we chose two random variables independently then the co-relation between them is said to be 0.

So, a full-rank matrix can be thought of as vectors(rows/columns) of independent random variables right? if yes then co-relation between such vectors must be 0 right?

can you give me more insight into this?

To support my argument, kindly have a look at the start of 2nd page in the following pdf.

https://www.stat.cmu.edu/~cshalizi/uADA/13/reminders/uncorrelated-vs-independent.pdf

Thank you

The correlation between two vectors is related to the inner product between them, the absolute value of which, when scaled by product of 2 norms of the individual vectors, results in value in closed interval of 0 to 1, in accordance with C.B.S Inequality, if the vectors involved are real vectors , the inner product itself lies in the closed interval -1 to +1 in that case, which is the expression for the correlation co-efficient between the two vectors.

Correlation between a set of linearly independent vectors relate to mutual angular separations among each other, a strong correlation implying closely aligned set of linearly independent vectors while in the case of a mutually orthogonal linearly independent set , we have situation of no mutual correlation. A linearly independent set of vectors define a co-ordinate system in an appropriate subspace of a vector space, and we know that orthogonal co-ordinate systems are not the only possible co-ordinate systems in a vector space.

When you talk of statistical correlation, a concept of averaging over a population , i.e the expectation value comes into picture, if for two random vectors X and Y are said to be uncorrelated in the above sense of inner product formulation , it means that the inner product of their numerical vector realizations, when averaged over the population , turns out to be zero, but for a particular numerical realization, the inner product can be any appropriate field scalar, with possibility of zero value if the sampled X and Y vectors turning out to be mutually orthogonal.

In other words, a linearly independent set of vectors, unless all mutually orthogonal, will have non zero inner products between some of the pairs of vectors belonging to the set, this value, can be appropriately related to cosine of an angle, which is not 90 degree in this particular case, hence, these pair of vectors are said to be correlated in that sense, since linear independence does not imply mutual orthogonality, a linearly independent set is in general a correlated set of vectors, with strength of correlation increasing as the vectors of set become more and more closely aligned while preserving the linear independence criterion.

In case of ill conditioned linearly independent sets, a small numerical perturbation introduced at one or more entries of the vectors involved in the set, can make the initial linearly independent set a linearly dependent one.

The word independent has two meanings in this context.

A row in a matrix is said to be an linearly independent row if that row in the matrix cannot be defined as a linear combination of the other rows. From the pdf you refer to, two random variables are statistically independent when their joint probability distribution is the product of their marginal probability distributions.

These are not the same things. The two concepts are linked since if vectors x and y are not linearly independent then they can also not be statistically independent. However, to answer your initial question, linear independence does not guarantee statistical independence, since it's possible to have covariance(x,y) non zero even if x and y are linearly independent.