These 'entropies' depend upon a parameter, which can be varied between two limits. In those limits they reduce to the Shannon-Gibbs and Hartley-Boltzmann entropies. If such entropies did exist they could be derived from the maximum-entropy formalism where the Lagrange multiplier would be identified as the parameter. Then, like all the other Lagrange multipliers, the parameter would have to be given a thermodynamic interpretation as an intensive variable which would be uniform and common to all systems, like the temperature and chemical potential. The Renyi and Havdra-Charvat entropies cannot be derived from the maximum-entropy formalism. Thus, there can be no entropy that can be parameter dependent, and whose parameter would be different for different systems.
It seems to me that the parameters in the Renyi and Tsallis entropies represent a best fit of a particular function (different for the two cases) to the empirically measured probability distributions of the available microstates. There is no uncertainty in the probability distributions of the Gibbs or the Boltzmann entropies, and hence, no extra parameters are necessary.
If the Renyi and Havrda-Charvat entropies are "real" entropies, in the sense that they can be applied to all systems with the same degrees of freedom, then their parameters must be universal in the sense that alpha and beta are universal, alpha=chemical potential/T, beta=1/T, and both are constants when thermal equilibrium and chemical equilibrium are achieved. Since the parameters in the above mentioned "entropies" cannot be given such an interpretation, they are not "real" entropies but, rather, interpolation formulas between "real" entropies. It would be impossible to conclude that the entropy of a system with a given parameter would be greater or less than that of the same system with another value of that parameter.
One can take an axiomatic approach to defining entropies. That is, we first define which properties seem natural and then home in on an entropy that has these properties. This is how the Renyi entropy originally came about. There are several properties that seem natural for an entropy / information measure:
1) positivity
2) invariance under permutations
3) additivity
4) continuity
5) an arithmetic mean property
All five properties above hold for the Shannon entropy, while 1-4 hold for the Renyi entropy. So this gives some sort of justification for the Renyi measure.
Furthermore, the Renyi entropies become operationally relevant in the setting of "one-shot" information theory, where the goal is to characterize the limitations on information processing tasks if given a single instance of a resource and an error tolerance. For example, the min- and max-entropies are special cases of the Renyi entropy and these characterize randomness extraction and compression, respectively, in the one-shot setting.
While the Renyi entropies are quite useful operationally, it is not clear to me that the Tsallis families have been shown to be useful in information theory.
For more reading, I think good references are
http://arxiv.org/pdf/1306.3142.pdf
http://arxiv.org/abs/1203.2142
Entropy: A concept that is not a physical quantity
https://www.researchgate.net/publication/230554936_Entropy_A_concept_that_is_not_a_physical_quantity
By 'continuity' I suppose we refer to property no. 1) of the list in Sec. II.1 of http://arxiv.org/pdf/1306.3142.pdf . In the language of quantum mechanics, this seems to be equivalent to 'Lesche's continuity' of statistical mechanics, i.e. small perturbations in the distribution of probability leads to small perturbations in the value of the entropy. Indeed, Tsallis and Rényi entropies share this property.
As fas as I know, the only property of Shannon's entropy which is not shared by Tsallis' entropy is additivity, namely property no. 4) of the list above. Tsallis' entropy of two statistically independent systems differs from the sum of the Tsallis' entropies of the two systems by an additive term. The latter term is the product of both entropies and a dimensonless number, usually written in the form q - 1 wher q is a real number in the range 0 < q < 2. If q -> 1 then Shannon's entropy is retrieved.
It is true that escort probabilities allow formal generalization of most results of the q = 1 statistical mechanics (including the central limit theorem, Jarzynski' equality, etc.) to the general case of q different from 1. Even Einstein's rule for deviations from a maximum entropy state has been demonstrated, for a particular monotonic function of Tsallis' entropy at least.
It is tantalizing that the constrained maxima of Tsallis entropy are described not by Gaussian probability distributions, as in the q = 1 case, but by q-exponential distributions; and the tails of q-exponential distributions behave as power laws, precisely like the distribution functions of most non-equilibrium steady state both in physics and outside it e.g. in econophysics).
Moreover, q-exponentials are the steady solutions of non-linear Fokker-Planck equations, just like Gaussian distributions are solutions of linear Fokker-Planck equations, and H-theorems ensure relaxation to steady states both for q = 1 and q different from zero under the same boundary conditions.
Finally, non-linear Fokker-Planck equations are the kinetic equations associated to nonlinear Langevin equations with Brownian motion, just like linear Fokker-Planck equations are the kinetic equations associated to linear Langevin equations with Brownian motion.
All these happy circumstances motivated a flood of publications ten years ago, with the goal of the Holy Grail of nonequilibrium thermodynamics in mind: the General Principle of Stability which should play far from equilibrium the same role of the Second Principle at equilibrium.
Trouble is, the Lagrange multipliers's unambiguously correspond to measured quantity only if q is known a priori. To date, there is no way to compute q for a give system once e.g. its Hamiltonian, its partition function -or something like that- is known.
To make thing worse for Tsallis' enthusiastic disciples, geometrical Brownian processes (which include e.g. Black-Scholem seminal model in econophysics and many problems of cellular growth in biology, newtork physics, droplet growth in hydrology and city growth in urbanistics) lead to lognormal distributions in steady state; the tails of the latter resemble power laws, just like the q-exponential distributions above. Actually, lognormal distributions maximize Shannon entropy, but with constraints which differ from those concerning the maximization of Shannon entropy which leads to a Gaussian distribution.
Finally, if a Lévy-stable process replaces Brownian motion even in the simplest Langevin equation, then the associated Fokker-Planck equation has fractional derivatives (in space only for Markovian process, in time too for non-Markovian ones); and if a relaxed, steady solution exists for such equations (as it has been shown for selected cases only to date) then its tail too exhibits power-law behaviour!
Even if Tsallis' disciples were successful, it would be difficult to understand wether their results may actually describe observations, as different models lead to similar predictions.
Today, people publish a lot about fractional Fokker-Planck equations, and Bernard Lavenda's question stands still, even if possibly reformulated as follows:
"why on Earth should we bother with entropies which differ from Shanon/Gibbs' entropy, if they contain free parameters we are not able to compute?"
What if we have several parameters? Then the situation can be described with fuzzy Shannon entropy ( see my paper in Journal of Physics & Astronomy, 2016- Approach with different entropies...)
In the situation with acceleration and expansion ( Universe, tokamak ), the problem is described by non-equilibrium.
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Thermodynamics