I read here that principal components scores are always in Euclidean distance and the distance of PCA is Euclidean: https://www.mii.lt/zilinskas/uploads/visualization/lectures/lect4/lect4_pca/PCA1.ppt
Is it true? I have a list of 20 principal components scores and have never been shown what distance measure do they represent. I want to calculate the Manhattan distance similarity and indicies between my samples according to these 20 principal components, but it would be pointless if the principal components are already made of Euclidean distance and I calculate out of them the Manhattan. So do always the PC scores actually represent the Euclidean distance measure or any else? Or they are not based on any distance measure? I hope that they don't, so I can go ahead and obtain the accurate Manhattan distance between the samples.
Theoretically speaking, PCA is based on orthogonal linear transformations, which only require an inner product space. Now, as an inner product induces a norm, and a norm induces a distance, then you can expect to have PCA on any space with a distance/norm that meets the axioms of a norm (not sure if you can always go back to define an inner product from a norm, though). Manhattan is one of such distances. Of course classical euclidean distance too.
However, all the implementations that I know assume an euclidean space (where all the typical operations are well-known, like computing eigenvectors for example), so, as far as I know, you would have to implement PCA from the grounds to use another distance. In the case of Manhattan, you should get into the so-called Taxicab geometry, in which the concept of inner product has been defined and so you can start with all the other linear algebra requirements for a PCA. This is heavy work! so I instead would suggest that if manhattan geometry is important for your analysis, then maybe you can think on alternative analytic procedures. You would have the same problems with CCA or MDS though, so I am thinking on, for example, clustering methods. With the typical k-means this problem arises again (assumption of euclidean space), so I am really thinking on K-medoids/PAM or Affinity Propagation clustering (apcluster package in R). The latter approach does not even require to specify in advance the number of clusters (a typical drawback of clustering methods). The reference for this development is this:
Frey, Brendan J., and Delbert Dueck. "Clustering by passing messages between data points." science 315.5814 (2007): 972-976.
and this method available for R:
Bodenhofer, Ulrich, Andreas Kothmeier, and Sepp Hochreiter. "APCluster: an R package for affinity propagation clustering." Bioinformatics 27.17 (2011): 2463-2464.
I hope that helps. Best regards.
Is your original data euclidean? If so then doing PCA first will not matter anyway.
PCA is always gives Euclidean distance as it is calculated based on variance, which is part of classical euclidean geometry and is the square of the distance between the data point and the origin. Manhattan distance is a step wise distance that depends on point to point spacing and connections rather than reference to a central origin.
Manhattan distance is more relevant where you believe your data has limited discrete possible outcomes. Euclidean distance is relevant where you believe your data comes from a continuous range of possible outcomes.
In many real world applications the distinction is fudged and there may not be a clear winner.
Also, using euclidean techniques first does not change the structure of the data, it just rotates it. To me that suggests that any discrete characteristics are retained so Manahttan distances can still be worked out after. It would be interesting to see what others think. However, transforming into Manhattan distance, as it assumes discreteness, may affect the data structure and so may disrupt subsequent euclidean based processing of the data.
Is the number of discrete possibilities increase or the variance between them starts to blur the discreteness then it looks more and more like a continuous situation.
- Badges
- Science topic
- Similar topics
- Agricultural Science
- Environmental Science