I'm working on comparing latent space representations of image patches which are encoded as multivariate normal distributions over the respective latent space.
Which metrics - besides (symmetric) KL-divergence, Hellinger distance and Bhattacharyya distance - exist to measure the distance between multivariate normal distributions, ideally fulfilling the mathematical definitions of a metric?
Second, from what I've noticed, Hellinger distance has a very small "window of sensitivity" - meaning that if I compute the similarities between encoded distributions, I get small values for identical image patches and values close to 1 for everything else, while symmetric KL divergence covers a wider range of values and also measures small distance between non-identical input. Any ideas on this?
You might try looking at the Continuous Ranked Probability Score (CRPS) . You would need the version of this that compares two probability distributions, and would also need to extend the definition from 1 to multiple dimensions. Essentially it is just the integral of the squared difference between the two cumulative distributions functions. I don't know if there are formulae for univariate or multivariate normal distributions. In the univariate case, it is an extension of the mean absolute error.
Otherwise, you might look through the list of distance measures at https://en.wikipedia.org/wiki/Statistical_distance .
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics