By definition, the capacity of a communication channel is given by the maximum of the mutual information between the input (X) and output (Y) of the channel, where the maximization is with respect to the input distribution, that is C=sup_{p_X(x)}MI(X;Y).
From my understanding (please correct me if I'm wrong), when we have a noisy channel, such that some of the input symbols may be confused in the output of the channel, we can draw a confusability graph of such a channel where nodes are symbols and two nodes are connected if and only if they could be confused in the output.
If we had to communicate using messages made out of single symbols only, then the largest number of messages that could be sent over such a channel would be α(G), the size of the largest independent set of vertices in the graph (in this case Shannon capacity of the graph equals independence number of that graph α(G)).
Does this mean that for such a channel, the maximum mutual information of the input and output of the channel (channel capacity) is α(G), and it is achieved by sending the symbols of the largest independent set?
Not clear what you are describing in this graph. But channel capacity is what the Shannon equation provides. The channel capacity in Shannon's equation is given as bits per second of information transferred without error. Simple equation:
Channel capacity (b/s) = Bandwidth (Hz) * logbase2(1 + S/N)
where S/N is signal to noise ratio expressed as watts/watts, not in dB.
This is a theoretical maximum channel capacity, and it incorporates any error correction schemes you might add. So, you have a tradeoff. Send fewer symbols per second, but extremely robust symbols, and you might get the same channel capacity as sending more fragile, but more spectrally efficient symbols, with a forward error correction code added in the stream.
Anyway, Shannon capacity and channel capacity are the same thing. Here is a short paper that seems to cover the subject matter more broadly:
http://large.stanford.edu/courses/2012/ph250/johnston1/
Hello Amirhossein Nouranizadeh You have an interesting question to discuss, but first you probably need to think again about your introductory text.
" when we have a noisy channel, such that some of the input symbols may be confused in the output of the channel "
It's not exactly like this. First of all, any channel is noisy, otherwise it's not real, or you do not need to communicate because everything is known with no uncertainty.
There is no such thing as the error-free zone of the encoding alphabet and the error-free zone of the decoding alphabet. Otherwise it means that the job has not been done.
Take the case you want to transmit one bit b. It takes values 0 or 1.
Assume that the sender sends b, the receiver receives b', at the other end of the tranmission channel.
The channel is noisy (otherwise it's not real...) then the probability that b'=b is less than 1: P(b'=b) 0
Clear?
That's how life is.
Now instead of sending a single bit b, you send a vector v, and you receive a vector v' (in channel language they speak of "words" but a word is avector, that's the same)
Then error probability is P(v' # v) = 1- P(v'=v).
You should not assume that there are immune vectors and contaminable vectors. If the channel coding is done properly, it spreads the risk evenly, usually.
There are other strategies. In the first Digital Mobile Communication Codec (GSM first generation) there was however a somewhat different structure:
-hierarchy of parameters (say projection of vectors v on subspaces V1, V2, V3, etc, for instance the first two characters of one words, the the following two, etc)
-coding with robustness herarchy
Protect V1 more than V2, protect V2 more than V3.
-decoding with hierarchy of error protection
Then decoding after transmission of v sent gives at receiver v', reconstructed from projections v'1, v'2, v'3. Then v'1 has lower probability of error than v'2, which has lower probability of error than v'3.
With such as scheme, you give more protection resource to what is more dramatic to lose.
Imagine it's the remote control of a car: going forward + or going - backward is more crucial information than the precision on the geometric angle of the movement. So you protect it more.
I hope that with the above you get a concrete sense of what is happening at a sender, on a channel, and at a receiver.
Does it help you?
- Badges
- Science topic