With EEG signals, wavelet (Shannon) entropy gave negative values. Can anyone explain why?
It is possible that you have one pj pretty close to zero, and then the MATLAB implementation of the log function misses accuracy.
I suggest just to try the following:
Add an small constant to all pj
(In choosing the constant, recall that MATLAB, for Shannon Entropy requires that
"the elements of probability vector P must sum to 1 +/- .00001.")
The constant should be small enough so the WE value is almost unaffected, but big enough to have a positive total WE.
Shannon entropy is never negative since it is minus the logarithm of a probability (Probability is always between zero and one).
- Hence - (log (any value between 0 to 1)) should be positive.
- Therefore, you have to use the Matlab function carefully as per requirement.
Jupitara Hazarika
Classic definition of Entropy is for discrete variables and it is always positive, but the extension of Shannon Entropy to the continues variables CAN BE NEGATIVE. This kind of entropy is called "differential entropy" and it is calculated as:
H[x] = 1/2 * (1 + ln(2 * pi * sigma^2)) ,
in which sigma is the variance of your random variable.
I'm not sure how you calculate entropy in MATLAB (discrete or continues) but this may explains why you get negative values.
Thank you everyone for your help.
I would like to add here that I found that MATLAB uses signal amplitude instead of probability to compute Shannon entropy. May be this is the reason for getting negative values..
A work around to this problem is that the probability p(n) in the shannon entropy expression for a signal x(i) can be calculated using the expression
p(n) = |x(n)|^2 / sum over i of x(i)^2
This ensures that the p(n)
- Badges
- Science topic