Dear Lazaros Moysis,

Here is the answer:

What is the maximum value of entropy in information theory?

Information and entropy can be measured in different units. If the unit is a bit the maximal entropy is log_2 (n) where log_2 denotes the logarithm with base 2. If the unit is a nat (natural unit) the maximum entropy is ln(n). If the unit is a digit the maximum entropy is log(n).

Hi Lazaros Moysis ,

If you are considering only the ASCII characters from 32 to 127 the maximum Shannon-entropy is 7 bits. Actually your can represent every character (95) in log(95)/log(2)=6.569 bits. It means you will never have to use 7 bits to represent the actual entropy of an ASCII text. However I recommend to use always the whole ASCII table (256 different characters) to represent the entropy of any text. If you are using the standard 8-bits entropy scale, you can compare ASCII text, images, encrypted data. However if you are using the 7-bits scale as you mentioned it is make no sense to compare an encrypted AES output with standard ASCII text (because they are measured in different entropy scale).

You can represent everything in the standard 8-bits entropy scale , however this is not true for the 7-bits scale. For example If you are encrypting a message and represent the result using only 7 bits, it is misleading, because actually you have more than 7 bits of entropy.

Most of the programs (MATLAB, PARI/GP, ent built-in linux command) are using the 8-bit (2^8) standard entropy scale, because the result is comparable. (no matter what is the actual maximum entropy)

I hope this help you a little bit.

Regards,

Norbert

The 8-bit entropy calculation is exactly useful for text encryption. Even, it is useful if we go for bit scrambling. While scrambling the bits, we can properly use all 8-bits. It is always better to write our code. It is a simple code.

If the value is 8, it has further advantage.

Several methods are there for determining the entropy of a text or its individual segments, called n-grams. Shannon method is one of the most popular method, which is based on the representation of the text by a Markov chain of depth n.

Second one is dictionary-based method for determining the entropy of n-grams, which combines the Kolmogorov’s combinatorial approach and a corollary of the Shannon’s second theorem, which gives an asymptotic estimate of the number of meaningful texts.