In PCA, for dimensionality reduction we select some eigen values.
Is there any relation between eigen value and covariance ?
In PCA under linearity assumption, main effort is to find New set of Basis Vectors (directions in simple terms) which maximizes SNR (or maximizes
the variance between axis) on new Basis Vectors. In this regard, eigen Values are the roots of characteristics equation formed using symmetric co variance matrix where as eigen vectors gives new basis vectors.
There are many good tutorials available and two good tutorial link
https://www.cs.princeton.edu/picasso/mats/PCA-Tutorial-Intuition_jp.pdf
http://dai.fmph.uniba.sk/courses/ml/sl/PCA.pdf
To add to Ashutosh Dwividi: It is a theorem that the eigenvalues
lambda_1 >= lambda_2 >= lambda_3 ... lambda_n>=0
of the covariance matrix are all non negative, and their corresponding eigenvectors can be chosen orthonormal (they are automatically orthogonal for different eigenvalues).
The eigenvector corresponding to the largest eigenvalue ("leading eigenvector) is the direction which gives the largest variation, the eigenvector corresponding to the second largest eigenvalue (subleading eigenvector) is the direction that gives the largest variation if you have taken the loading of the leading eigenvector into acount (i.e. the variation not explained by the leading eigenvector), the eigenvector of the sub sub leading eigenvector, is direction that give the largest variation if you take the leading and subleading eigenvectors into account (i.e. the variation unexplained by the leading and sub leading eigenvector) etc.
- Badges
- Science topic
- Similar topics
- Psychology