Dear Muhammad Ahsan Zamee,

In the linear algebra the SVD is a factorization of a rectangular/square real or complex matrix analogous to the digonaliztion of symmetric or Hermitian square matrices using a basis of eigenvectors. SVD is a stable and an effective method to split the system into a set of linearly independent components, each of them bearing own energy contribution

The SVD is an important tool in several different applications such as:

The SVD can be used to image compression. Digital image is represented by a matrix and the entries are the pixel values, the SVD is employed to the image matrix, by delete some of nonzero singular values and keep only the first k large singular values out of the r nonzero singular valued, the image dimension can be reduced

With the SVD you can show that a MIMO channel with M antennas at each side (M at the transmitter and M at the receiver) can be treated as M separate single-antenna channels. Hence, one can transmit information over the MIMO channel up to M times faster than with a single-antenna channel.

Note that "antennas" can be replaced by "cables" or some other kind of transmission medium

Dear Muhammad Ahsan Zamee,

In the linear algebra the SVD is a factorization of a rectangular/square real or complex matrix analogous to the digonaliztion of symmetric or Hermitian square matrices using a basis of eigenvectors. SVD is a stable and an effective method to split the system into a set of linearly independent components, each of them bearing own energy contribution

The SVD is an important tool in several different applications such as:

The SVD can be used to image compression. Digital image is represented by a matrix and the entries are the pixel values, the SVD is employed to the image matrix, by delete some of nonzero singular values and keep only the first k large singular values out of the r nonzero singular valued, the image dimension can be reduced

With the SVD you can show that a MIMO channel with M antennas at each side (M at the transmitter and M at the receiver) can be treated as M separate single-antenna channels. Hence, one can transmit information over the MIMO channel up to M times faster than with a single-antenna channel.

Note that "antennas" can be replaced by "cables" or some other kind of transmission medium

When a matrix is multiplied with a vector, it does do things 1. rotation 2. scaling. you can easily visualize it by multiplying a two dimensional vector with a 2x2 matrix (y=Ax) and see that y is a rotated and scaled version of x. When the dimension increases then you can't visualize it. SVD kind of helps u in finding out the the basis for the axis of rotation (U and V*) and singular value matrix (S) says the importance of that basis. So from SVD of matrix A, u can say that how it will rotate or scale the x.

The straight forward application is PCA or dimension reduction. If u can find that in certain axis there is no change then u can simply ignore that because they are not important. You can link the SVD to eigen value decomposition where eigen value decomposition is only for square matrix where as the SVD is valid for all the matrices.

This is my intuition to SVD and hope it helps.

BW

To understand Singular value decomposition, we need to consider the fact that there exist a 1-1 correspondence between the row space and column space of a matrix A(m by n), both of which are of dimension =r, where r=rank(A).

The question is how to relate a basis in Row space to one in column space and vice versa, Full rank factorization generate from a matrix A(m by n), rank=r, two matrices Q (m by r) and R(r by n), the rows of R forms a basis for Row space of A while columns of Q forms a Basis for Column space of A, SVD is a special case of the Full rank factorization where the row space basis under consideration is an orthonormal basis , which is the eigenbasis of AT*A considering only those eigenvalues of AT*A non-zero (in case AT*A is singular) and the column space basis in svd is the orthonormal eigenbasis of A*AT, considering only those eigenspaces associated with non zero eigenvalues of A*AT. Here AT stands for Transpose of A matrix.

colsp(AT*A) = colsp(AT)= rowsp(A) & colsp(A*AT)= colsp(A), colsp and rowsp stands for the column space and rowspace respectively of the matrix argument within () brackets.

The advantage of this particular orthonormal factorization of the two spaces is the ease of relating these two basis sets through A and AT, A transforms this svd Row space basis to the svd Column space basis with each vector amplified by the singular value associated with the pair of 1D subspaces (1D subspace in Row space and its counterpart 1D subspace in column space related by A and AT), similarly AT transforms the SVD column space basis to the svd Row space basis amplifying the individual vectors by their corresponding singular values. This form of relationship between the two spaces allow A( and also AT) to be expressed as sum of rank 1 matrices, amplified by the respective singular values, this allows to select suitable number of such components based on desired level of approximation and one can then achieve data dimensionality reduction.

Thank you everyone for your rigorous and highly effective help. It means a lot to me.

a very beautiful explanation on SVD can be seen on:

https://en.wikipedia.org/wiki/Singular-value_decomposition