In the paper "Dimensionality Reduction:A Comparative Review" indicates that PCA cannot handle non-linear data.
It is simply that the assumptions underlying PCA are linear - and the interpretation is only valid if those assumptions are true. OF course, you can still do a PCA computation on nonlinear data - but the results will be meaningless, beyond decomposing to the dominant linear modes and provided a global linear representation of the spread of the data.
Thanks for your answer Professor. Where can I find these principle assumptions behind PCA and some such dimensionality reducers?
PCA does much less than people think it does. It really just provides a rotation of the representation axis system that optimizes the variance of the given data in the least number of dimensions. It does not pick out any structure at all; it is sensitive only to variance in the data. I use it to eliminate redundant (zero-variance) dimensions and then use something like Linear Discriminant Analysis to gain insight into what the data is saying for my application. Nonlinear correlation can be used to discern structure. You apply a nonlinear operator on the data and then do normal linear correlation. If the nonlinear operator matches the actual nonlinearity, the correlation will shoot up high.
Dear Kushal,
PCA indeed works optimally only in the situation where the correlations are linear, which is most of the time an approximation. You may want to build a dummy data set with strongly non linear correlations and check for the result. If your data are non linear, I see two possibilities:
1) Try a transformation that will make them somehow linear
2) Use a non-linear dimension reduction
http://en.wikipedia.org/wiki/Nonlinear_dimensionality_reduction
I am aware that using PCA blindly would cause much less work and pain, but for the sake of good science, take care of non linearities!
Good luck,
Xavier
PCA simply performs a rotation of the given coordinate axes. Rotation is a linear operator. But the axis frame returned by PCA (the eigenspace) captures every bit of the variance of the original data. What you would like to do, however, is to drop axes with small eigenvalues (weights) and proceed on a smaller data set. If your data has nonlinear structure, as most does, then PCA will have a larger number of dimensions with nonzero weights. And the objective of PCA is to *minimize* the number of dimensions with significant weight.
PCA works better for linear data and has limitations with nonlinear data. For nonlinear data, t-SNE can be useful
Read more about PCA https://www.reneshbedre.com/blog/principal-component-analysis.html Read more about t-SNE https://www.reneshbedre.com/blog/tsne.html
- Badges
- Science topic