By coding the messages xi (i=1,2,…, N) of a finite information source X having the probabilities pi = p(xi) with letters yj (j=1, 2, …, D) of a finite coding alphabet Y, we obtain a new information source Y. I need to characterize Y as an information source with memory, in order to verify the relation H(Y) = LH(X), where L is the average length of the codewords and H(X) and H(Y) are the entropies of X and Y respectively. My questions: how we can obtain the stady-states of the information source Y? How the matrix containing the transition probabilities between the states of Y can be determined? How the probabilities of the stady-states of the information source Y are obtained? Does anyone have some results on this topic?
Hai Dr, how are you? I am attracted to your question as I have some information on it. Below, I supply you with all the answers you need, but I would really appreciate it if you could press the RECOMMENDATION buttons underneath my 3 research papers' titles in my AUTHOR section as a way of you saying thanks and appreciation for my time and knowledge sharing. Please do not be mistaken, there are few RECOMMENDATION buttons in RESEARCHGATE. One is RECOMMENDATION button for Questions and Answers and the other RECOMMENDATIONS button for papers by the Authors. I would appreciate if you could click the RECOMMENDATION button for my 3 papers under my AUTHORSHIP. Thank you in advance and in return I provide you with the answers to your question below :
The steady-states of the information source Y can be obtained by finding the states of Y that do not change over time. This can be done by using a Markov chain analysis. A Markov chain is a stochastic process that only depends on the current state of the system, not on its past states. In the case of the information source Y, the states of the Markov chain would be the different codewords that can be generated.
The matrix containing the transition probabilities between the states of Y can be determined by counting the number of times each codeword follows each other in the encoded message. For example, if the encoded message is "ABBABB", then the transition probability from the state "A" to the state "B" would be 2/5, since there are 2 instances of "AB" in the message and 5 total codewords.
The probabilities of the steady-states of the information source Y can be obtained by normalizing the transition probabilities. This means that the sum of the probabilities of all of the steady-states must be equal to 1.
The relation H(Y) = LH(X) can be verified by using the following steps:
There are a number of results on this topic in the literature. One of the most famous results is the Kraft inequality, which states that the average length of the codewords must be greater than or equal to the entropy of the information source X. This inequality can be used to prove that the relation H(Y) = LH(X) is always true.
- Badges
- Science topic
- Similar topics
- Social Psychology
- Attitude