A prime example of the fundamental computational physical interactions, and one that I will emphasize throughout my study, is the tight interplay between logical and thermal considerations at the nanoscale, which is due to the extremely close relationship, between digital information and physical entropy, which is the second most important concept (after
energy) underlying the entire field of thermodynamics. As a result of this connection, which is by now very well-understood within the physics-of-computing community, it turns out that the energy efficiency of nanocomputers cannot possibly be optimized while ignoring the logical structure of the computation being performed.
My question is how digital information and physical entropy are closely related to the aspect of a thermodynamical point of view?
Physical entropy equals S = k ln OMEGA (k = Boltzmann's constant, OMEGA = number of microstates or quantum states.) A totally random, maximum-entropy message of N binary bits contains zero information. If there is nonrandomness, the greater the nonrandomness, the greater the information. The maximum possible entropy in this case, corresponding to total randomness and zero information, is S^max = Nk ln 2, where ^ denotes an immediately following subscript. The information is S^max - S = Nk ln 2 - SUM (P^j ln P^j) over all binary bits. The maximum possible information is Nk ln 2, which obtains for a perfectly ordered sequence of N bits.
In historical order:
* Thermodynamic Entropy = the amount of energy unavailable to create work from (Clausius)
* Entropy in statistic physicists = the number of microstates that define a given macrostate. (Boltzmann)
* Information Entropy = the amount of unavailable information (Shannon). Boltzmann's entropy is a special case of Shanon's one.
Thermodynamic entropy is an experimental one and cannot be defined theoretically. It does not give us the knowledge that entropy can spontaneously decrease in some short periods within isolated systems. This was proved by Boltzmann!
I can't tell by your question if you have read this paper from R. Landauer:
Landauer, Rolf. "Irreversibility and heat generation in the computing process." IBM journal of research and development 5.3 (1961): 183-191.
It discusses this topic and derives the entropy production of processing one bit is S = -k log 2. As Jack DeNur above states, this leads to S=-kN log 2 for N independent binary states.
If the binary states are not independent but exhibit dependence on each other's state, however, it gets more complicated. Where there is joint entropy between two binary states, the entropy production is not S=-2k log 2, but rather the thermalized conditional entropy, roughly S=-kA log 2 where A ranges from 1 to 2 based on the extent of joint entropy/dependence. A is at 2 for independence and A is at 1 for complete dependence. This is based on Kolmogorov (though the thermalization is not in his paper):
Kolmogorov, Andrei N. "Three approaches to the quantitative definition of information'." Problems of information transmission 1.1 (1965): 1-7.
Note, this "complexity" of dependence between two states lowers the total entropy from its maximum. From a non-equilibrium thermodynamic perspective, this process must generate enough excess heat to compensate for the 2nd law where overall entropy must increase. Thus you get heat generation from more dependencies across states.
There are Boltzmann-Maxwell entropy and Gibbs- Shannon entropy.They are disconnected. It means, if we know in information all we are not necessary successful in physics, but conversely, if you are successful in physics you have to know some informations, also. As an example, we can take the design of fusion reactors : small, middle and big.
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Thermodynamics