Entropy definition through 1/p(x) is heuristic. In my opinion, interpretation of 1/p(x) as the measure of "surprise" is a simplification permitting the readers more the less acceptable understanding of the sense of entropy. Reality is more complex. For the beginning it is enough to try to give a definition of the uncertainty. Its further formalisation would be better to start from Hartley's definition, for p(x)=1/ n, it is closer to intuitive than through 1/p(x). Shannon (in the same work) extended Hartley's result to non-uniform distributions using asymptotic equipartition property of long sequences. This way is more complex but gives deeper understanding of particularities.

Addendum: Mean amount of information delivered by events yk about events xj is I(X,Y) = H(X)-H(X|Y). H(X) determines the prior uncertainty of the set of events X. H(X|Y) is the posterior (residual) uncertainty of these events after registration of some yk from the set Y, and I(X,Y) is equal to the removed uncertainty of events xk. If registration of yk unambiguously determines corresponding xk, i.e. p(xi|yk)=1 for i=r, and p(xi|yk)=0, for i not = k, then H(X|Y)=0 and I(X,Y) = H(X).

Thus, H(X) is amount of information delivered by the events yk unambiguously connected which the events xj.

Examples: yk are the signals received at the output of ideal channel or ideal (no errors ) measurement of random values.

Any noise makes H(X|Y)>0 and diminishes I(X,Y).

Sometimes this consideration is reduced to the extremely simplified form: H(X|X)=0 always, then H(X) = I(X,X) and entropy is equal to the amount of information in the event about itself. In the lectures, I prefer the first approach. The second one is too artificial - in practice, we cannot speak "if xk = xk, then...". We may say this only if we register xk without errors using ideal channels or measurement devices.

In my opinion, entropy IS a true measure of information; the axiomatic approach of Shannon is, surprisingly, not really discussed in standard textbooks on information theory (such as the one by Gray or the one by Cover & Thomas). However, you will find the axioms for example in Papoulis's book "Probability, Random Variables and Stochastic Processes". Indeed, there are just three very intuitive axioms what you would require of a measure of information, and these uniquely determine the entropy formula (up to a constant factor). As far as I know, these three axioms differ from those Shannon gave; in particular, his ominous third axiom is missing there.

If you are interested in a brief discussion of the axiom, refering to original literature I recommend

http://arxiv.org/abs/1106.1791 (A Characterization of Entropy in Terms of Information Loss)

and

http://arxiv.org/abs/1109.6440 (Extropy: a complementary dual of entropy).

In particular, the latter refers to a book of Jaynes, which very critically discusses Shannon's third axiom. (Jaynes was famous for employing the maximum entropy method in statistical physics, but often argued about Shannon's results.)

The Shannon notion of information has its roots in a more general measure referred to as the "Diversity Index". One specific case of the diversity index is mapped to Entropy. (search Wiki for the article)

For applications in computer science and telecommunications,entropy seems to be a good measure. Intuitively, for a finite sample space, the number of possible events is exponential in the cardinality number of the set and inevitably, we should look for a logarithmic measure to damp the exponential. ;)

Semantic information can be defined by I=log(Truth_value/Logical_probability)=testing_severity-relative_deviation 

This formula accords with Popper's information criterion for scientific advances.

For example, consider  information the forecast of "There is rainstorm tomorrrrow". Its logical probability is about 0.1. if it is true, true-value is 1, and  I=log10.

If it is half true, true-value is 0.5 and  I=log5.

If it is false, so that true-value

I have the feeling that the entropy formulation is an actual measure of the minimum number of bits ( 0/1) needed to characterize and I/O signal. Normalized on the entire population. I wonder if this "feeling" is correct and ifn case if someone here can give a mathematical formulation. If Log2(p) is the number of digits needed to store the p information (?) and p + N is the number of times the info appears.... the SUM (p*log) could be a representation of the total number of digits needed... Am I totaly out of track here? thx and sorry for the trivial question....

H(X) is potential information. I(X;Y)=H(X)-H(X|Y) is conveyed informtion.

Average log( normalized likelihood) is information that is actually received.