In information theory, the entropy of a variable is the amount of information contained in the variable. One way to understand the concept of the amount of information is to tie it to how difficult or easy it is to guess the value. The easier it is to guess the value of the variable, the less “surprise” in the variable and so the less information the variable has.
Rényi entropy of order q is defined for q ≥ 1 by the equation,
S = (1/1-q) log (Σ p^q)
http://www.tina-vision.net/docs/memos/2004-004.pdf
As order q increases, the entropy weakens.
Why we are concerned about higher orders? What is the physical significance of order when calculating the entropy?
You may look at other entropies, search articles from Prof. Michèle Basseville on entropy (of probability measures)
Very simply said. Renyi entropy is derived from Boltzmann entropy by simplifications imposed on microstates. You get a resulting distribution of probabilities of states.
This distribution changes with changing parameter q. For a certain value of the parameter q in complex systems, you get the maximal value of Renyi entropy. This value is equal to q = 1/(1 + K).
At this point, Renyi distribution becomes scale-free one for complex systems.
Read more about all derivations here
https://www.researchgate.net/publication/243167792_Renyi_entropy_as_a_statistical_entropy_for_complex_systems
There is a preprint for the paper in ArXive.
You can decide in this way, whether the system under observation is complex or not in such way.
You may look at other entropies, search articles from Prof. Michèle Basseville on entropy (of probability measures)
I feel we can remind that the Rényi entropy is important in ecology and statistics as index of diversity.
- Badges
- Science topic
- Similar topics
- Engineering
- Electrical Engineering