I want to compare two or more entropy measures. I found many research paper compared them with the help of their performance w.r.t. linguistic variables by taking a numerical example. But the performance changes with values, so we can't generalize.
Why would you want to compare two entropy measures? I assume that most, if not all, entropy measures should yield similar results, though at different scaled values. My experience, for example, suggests that Tsallis entropy will yield a higher value than Shannon entropy. I might be very wrong, though.
Tsallis entroy only gives a smaller value than Shannon entropy when the parameter (order) is less than one.
To compare entropy measures, you can generalize with Rényi entropy, by appropriately changing the Rényi's exponent according to the type of data you got. The higher the Rényi exponent, the most probable value of entropy you obtain for the most common probability in your set.
see also my:
https://www.researchgate.net/publication/271204916_Multifractal_exponents_and_Rnji_entropy_a_clue_for_brain_function
Even it my paper refers to brain function and multifractals, the concept is easily generalizable to many probability distributions, in countless different fields.
Preprint Multifractal exponents and Rénji entropy: a clue for brain function?
Entropy: A concept that is not a physical quantity
https://www.researchgate.net/publication/230554936_Entropy_A_concept_that_is_not_a_physical_quantity
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Bioinformatics
- Bioinformatic Tools