I believe that in this case, the answer to your question depends on the type of entropy you are asking about. Entropy in a thermodynamic system is a bit different than entropy in an information-theoretic sense. Thermodynamics can tell us about the macrostate— or thermodynamic probability— of any thermodynamic system, which would include system-wide variables such as its pressure, temperature, volume and (thermodynamic) entropy. However crucial they are at a macro level, thermodynamic principles are insufficient when attempting to understand the various potential microstatesof a given system. Microstates are the various different ways could the molecules of the given system be positioned or arranged while maintaining the overall values of the macrostate.

From a Statistical Mechanics perspective, we begin with the Gibbs conception of entropy, but concern ourselves with Boltzmann’s conception of entropy. Boltzmann defined entropy as a way of measuring all of the possible microstates which could yield a given macrostate. Therefore, the absolute entropy of a macrosystem can be found by multiplying Boltzmann’s constant by the natural log of the number of possible microstates. I think that it is a bit of a misnomer to describe entropy as “disorder” when it has been better described (by Boltzmann, Jaynes and others) as “a lack of information” about a given system.

If we necessarily leave behind the quantity of heat, as we must with many non-thermodynamic utilizations of the idea of entropy, the concept of information entropy takes shape. Claude Shannon borrowed the terminology and developed a probability density schema which measures the uncertainty of information, or average information received, in a communication. Some physical scientists might find this a problematic analogue, but Jaynes has argued that it is a direct and useful application of the thermodynamic principles. On that question, I will allow you to decide!

To your point, I’ll briefly provide a working definition of chaos. Chaos— or, deterministic chaos— occurs in dynamical systems which, according to Robert Devaney, are a) sensitive to initial conditions, b) topologically mixing, and c) contain dense periodic orbits. So, however much a chaotic system is deterministic, it is not likewise predictable. In other words, we should theoretically be able to predict its trajectory, but after several iterations this becomes impossible. The state of the system rapidly diverges from its known state over time and becomes, practically speaking, unknowable.

So, getting back to your question “how are chaos and entropy related,” you are probably beginning to see how various measures of entropy might be useful in “diagnosing” chaos, if you will. In particular, Kolmogorov-Sinai entropy (KS entropy or metric entropy) as our colleague Vitor Martins de Oliveira so elegantly explains above, is a crucial measure which is employed as an indicator of chaos in a given system. I might differ from our colleague slightly in the way that I explain topological entropy, which— although it shares a significant relationship to KS entropy, set forth in what is known as the variational principle— it additionally requires that the dynamical system occupy a compact Hausdorff topological space. In this case, the topological entropy of a dynamical system can be seen as the average exponential growth rate of the distinguishable orbits over all iterations, and can be used to measure the complexity of the system, not necessarily chaos, however positive topological entropy implies Li-Yorke chaos.

Sundarapandian Vaidyanathan, thank you for posting an excellent and thought-provoking question!

Dear Remi Cornwall - You have detailed well entropy and chaos concepts separately, and I thank you for the same.

My question really is this - How are entropy and chaos related?

Entropy: A concept that is not a physical quantity

https://www.researchgate.net/publication/230554936_Entropy_A_concept_that_is_not_a_physical_quantity

Dear Sundarapandian Vaidyanathan,

You can define a metric entropy (KS entropy) or a topological entropy and both can be used as chaotic indicators by themselves.

In special, the metric entropy is based on Shannon's entropy, which is largely used in information theory. Following the works of Young (1982) and Kantz and Grassberger (1985), one can relate the metric entropy of an ergodic invariant measure to the lyapunov exponents of an orbit of the system by defining the information dimension. Therefore, there is a directly link between entropy and chaos.

I kindly suggest to you the book "Chaos on Dynamical Systems" by Edward Ott (2002).

Kind regards,

Vitor

See the book "Second Law" written by Peter Atkins for a deep understanding of this subject.

Regards

I believe that in this case, the answer to your question depends on the type of entropy you are asking about. Entropy in a thermodynamic system is a bit different than entropy in an information-theoretic sense. Thermodynamics can tell us about the macrostate— or thermodynamic probability— of any thermodynamic system, which would include system-wide variables such as its pressure, temperature, volume and (thermodynamic) entropy. However crucial they are at a macro level, thermodynamic principles are insufficient when attempting to understand the various potential microstatesof a given system. Microstates are the various different ways could the molecules of the given system be positioned or arranged while maintaining the overall values of the macrostate.

From a Statistical Mechanics perspective, we begin with the Gibbs conception of entropy, but concern ourselves with Boltzmann’s conception of entropy. Boltzmann defined entropy as a way of measuring all of the possible microstates which could yield a given macrostate. Therefore, the absolute entropy of a macrosystem can be found by multiplying Boltzmann’s constant by the natural log of the number of possible microstates. I think that it is a bit of a misnomer to describe entropy as “disorder” when it has been better described (by Boltzmann, Jaynes and others) as “a lack of information” about a given system.

If we necessarily leave behind the quantity of heat, as we must with many non-thermodynamic utilizations of the idea of entropy, the concept of information entropy takes shape. Claude Shannon borrowed the terminology and developed a probability density schema which measures the uncertainty of information, or average information received, in a communication. Some physical scientists might find this a problematic analogue, but Jaynes has argued that it is a direct and useful application of the thermodynamic principles. On that question, I will allow you to decide!

To your point, I’ll briefly provide a working definition of chaos. Chaos— or, deterministic chaos— occurs in dynamical systems which, according to Robert Devaney, are a) sensitive to initial conditions, b) topologically mixing, and c) contain dense periodic orbits. So, however much a chaotic system is deterministic, it is not likewise predictable. In other words, we should theoretically be able to predict its trajectory, but after several iterations this becomes impossible. The state of the system rapidly diverges from its known state over time and becomes, practically speaking, unknowable.

So, getting back to your question “how are chaos and entropy related,” you are probably beginning to see how various measures of entropy might be useful in “diagnosing” chaos, if you will. In particular, Kolmogorov-Sinai entropy (KS entropy or metric entropy) as our colleague Vitor Martins de Oliveira so elegantly explains above, is a crucial measure which is employed as an indicator of chaos in a given system. I might differ from our colleague slightly in the way that I explain topological entropy, which— although it shares a significant relationship to KS entropy, set forth in what is known as the variational principle— it additionally requires that the dynamical system occupy a compact Hausdorff topological space. In this case, the topological entropy of a dynamical system can be seen as the average exponential growth rate of the distinguishable orbits over all iterations, and can be used to measure the complexity of the system, not necessarily chaos, however positive topological entropy implies Li-Yorke chaos.

Sundarapandian Vaidyanathan, thank you for posting an excellent and thought-provoking question!

The entropy production rate is related with the Lyapunov exponents spectrum. Lyapunov exponents are standard measures for chaoticity.

Some useful links:

http://homepages.vub.ac.be/~gaspard/G.ACP.07.pdf

http://www.williamhoover.info/Scans1990s/1994-1.pdf

I believe that the entropy change will lead to nonlinear behavior and equations, and chaotic is also a nonlinear behavior whose governing equations are also non-linear, which links these two categories together. So that is, increasing entropy causes non-linear behavior that is equal to chaotic .

You may find some discussion on the following link:

https://socratic.org/questions/how-does-entropy-relate-to-chaos-theory

Rényi entropy and its difference cases are explained here

https://en.wikipedia.org/wiki/R%C3%A9nyi_entropy