If they are equivalent, does the entropy calculated using the Shannon formula follow the law of entropy increase for macroscopic physical processes? If not, what is the difference between them?
No, thermodynamic entropy and Shannon entropy are not equivalent, although they share certain conceptual similarities.
Thermodynamic entropy, often denoted as S, is a concept from classical thermodynamics that measures the degree of disorder or randomness in a physical system. It is related to the number of microscopic states that a system can occupy, given its macroscopic properties such as energy, volume, and particle number. The second law of thermodynamics states that the total entropy of an isolated system tends to increase or remain constant over time.
On the other hand, Shannon entropy, named after Claude Shannon, is a concept from information theory that quantifies the amount of uncertainty or information content in a message or data source. It measures the average amount of "surprise" or "bits" needed to represent or transmit a message. In the context of information theory, entropy is often denoted as H.
While both thermodynamic entropy and Shannon entropy involve measuring degrees of disorder or uncertainty, they operate in different domains and have distinct mathematical formulations. Thermodynamic entropy is concerned with physical systems and is described by the laws of classical thermodynamics, while Shannon entropy deals with information and is based on probabilistic concepts.
However, there is a deep connection between the two concepts. In certain cases, Shannon entropy can be used to analyze and describe thermodynamic systems, particularly in statistical mechanics and the study of Boltzmann entropy. This connection is known as the "information-theoretic interpretation of thermodynamics" and provides insights into the relationship between entropy, information, and the microscopic behavior of physical systems.
Thank you for your answer, however, I am still unclear under what specific case Shannon entropy can be used to analyze and descript thermodynamic systems. Here is my case, where I am studying an irreversible and spontaneous macroscopic geological process, and the entropy of that system was calculated using the Shannon formula, we found that its entropy values increase with time, does that implies that this process follows the second law of thermodynamics, the principle of increase of entropy.@Yazen Alawaideh
Shannon himself explains the origin of his use of the term "entropy":
“My greatest concern was what to call it. I thought of calling it ‘information’, but the word was overly used, so I decided to call it ‘uncertainty’. When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, ‘You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage".
Tribus M., McIrvine E.C. Energy and information. Sci. Am. 1971;225:179–188. doi: 10.1038/scientificamerican0971-179.
Thermodynamic entropy and Shannon entropy are conceptually equivalent: the number of arrangements that are counted by Boltzmann entropy reflects the amount of Shannon information one would need to implement any particular arrangement(Bekenstein,2003). Futher support for their equivalent comes from Landauer (1961), and some experimental studies have also demostrated this equivalence.
- Badges
- Science topic
- Similar topics
- Psychology