Binary sequences 1100100101
Symbolized by binary P(0)=1/2 P(1)=1/2 H(x)=-0.5*log2(0.5)-0.5*log2(0.5)=-1
Symbolized by quaternions
P(11)=1/5 P(00)=1/5 P(10)=1/5 P(01)=2/5 H(x)=-1/5*log2(1/5)*3-2/5*log2(2/5)=-1.9219
...
Is there a problem with my understanding?
If not ,which result is information entropy?
Firstly, the calculated entropy in both cases is positive (entropy is always non-negative).
Secondly, the probability of each bit/symbol is calculated over a large number of occurrences.
Thirdly, entropy = 1 for 'Symbolized by binary' case means the entropy is maximum because each bit is equi-probable. On the other hand, for the 'symbolized by quaternions' case, the entropy < 2 because the calculations considers that all states are not equi-probable.
I hope it helps.
Danish Ilyas Thank you for your answer."The data compression limit is information entropy".Like the two entropies I pointed out.Which one is the entropy of the data compression limit？I can't understand.
Both calculations show entropy. The first case is for a source which is generating bits with equal probabilities (i.e. p(0)=p(1)=0.5). This source has maximum randomness and therefore maximum entropy. Therefore, you cannot use less than 10 bits to represent 10 bits from this resource.
The second case is for a source which is generating pairs of bits {00,10,01,11} with different probability for each. This resource does not have maximum randomness and therefore its entropy is less than 2 (2 is the maximum entropy which can be achieved by a pair of bits). So both calculated entropies (they should be non-zero values) are respective compression limits of different resources.
Danish Ilyas I understand. But both cases are the same source. Why say different sources. The source is “1100100101”.
Binary sequences, such as 1100100101, can be symbolized using quaternions. However, it's important to note that quaternions are typically used to represent rotations in three-dimensional space and are not commonly used for binary sequence representation. Nonetheless, if you want to symbolize a binary sequence using quaternions, you could assign quaternion values to each bit in the sequence.
Let's assume we use the following mapping for the quaternion values:
0 -> (1, 0, 0, 0) 1 -> (0, 1, 0, 0)
Using this mapping, the binary sequence 1100100101 can be symbolized as follows:
(0, 1, 0, 0), (0, 1, 0, 0), (1, 0, 0, 0), (0, 1, 0, 0), (1, 0, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (1, 0, 0, 0), (0, 1, 0, 0), (1, 0, 0, 0)
Each bit in the binary sequence is represented by a quaternion with the corresponding mapping. However, please note that this is an arbitrary mapping and may not have any specific meaning or widely recognized usage in the context of quaternions.
Aman Prabhakar Thank you for your answer.My purpose is data compression. What I want to understand is which entropy is the limit of data compression.Although quaternions are not commonly used, they can be used for data statistics and analysis.
- Badges
- Science topic
- Similar topics
- Counseling Psychology
- Confusion