To diagonalize a covariance matrix 500+x500+ is extremely difficult, almost impossible. To maximize a mean squared projection of the vectors on some direction a=(a1,a2,.....a500+) You are to solve a systems of 500+ nonlinear equations. What is the rest? Heuristic methods like genetic algorithm, ant colony, simulated annealing etc ?
I wonder when people easily say: "We've considered first 2/3/4.. PCs!"
Of course without any further information this is hard-but, in fact, it’s, also, meaningless and a waste of computer and human resources. It just doesn’t make sense to compute principal components of large unstructured matrices as an end in itself-it's a means towards something else, that, in fact, imposes additional constraints, that lead to non-trivial simplifications, which are what one is really after. What does makes sense is to ask what are the properties of covariance matrices that are of some particular interest. As usual, both in physics and mathematics, the symmetries come first-and these are crucial for the implementation of any numerical algorithm, anyway. What's unfortunate is how often the significance of the symmetries of the problem gets forgotten.
(The claim that the problem doesn't have any symmetries is wrong... The symmetries define the problem!)
Thank You very much, dear Stam for this Your answer. My colleagues deal with chemical sensors in cyclic temperature modulation regime and they use PC-analysis for selectivity purposes. The 500+ signal points series each from one of ~30 cycles they consider as the set of ~30 500+-dim.vectors and calculate PC1,PC2 (in what way?). Then they build polynom.(n=3) regression PC2 on PC1 for various analyte concentrations which serves the selectivity tool for the analyte. I have a lot of doubts that they manage to find these PCs. This is why I have asked the question.
Indeed what's needed is a model for what selectivity actually means. That's the problem with doing data analysis without discussing the physics-or chemistry-that ought to be of interest. I have the impression that the data analysis is used as a substitute or proxy for the model-of the chemical sensor. But it's the model of the chemical sensor itself that ought to be the focus of the study, I would've thought...Once that model is described, it can be used to deduce properties such as the selectivity... It's the model of the chemical sensor that ``compresses'' the information and makes the calculations, then, manageable-and, certainly, comprehensible. ``Long had their imagination sought the food of knowledge"" (E. Onegin) :)
Stam, and what about so called singular decomposition of nonsquared matrix X=(xij), i=1..30, j=1..500+ composed of 500+-dim vectors?
- Badges
- Science topic
- Similar topics
- Vectorization