I wrote a algorithm based on GMM. After a few iterations the following problem is caused: density values in Some observations are zero and the entire algorithm makes trouble.
For example, for i=67203 from observations, the density values in all components is zero (not even a small number).
With this code, I calculated the density of all components for this observation and all became zero.
for l=1:9
pr(l)=mvnpdf(y(67203,:),mu(l,:),sigma(:,:,l))
end
Hence, posterior probabilites and then parameters (mu and sigma matrix) are NaN.
Is there a way to prevent this zero values for densites and NaN values for Posteriori probabilities?
Hi Vu Nguyen,
Variance matrix obtained in estimation process of EM algorithm. there are out of my control.
In order to get this divide by zero problem, EVERY element of your summation would have to be zero, which seems extremely unlikely. I think you have a coding error. Note that there is no problem if the numerator is zero!
Hi Morton E. O'Kelly,
Thank you for response. i checked again parameters. i did not discover confusion in codes.
I wrote Sigma matrix as follows which can be see. According to the formula in attach file.
I think that is correct.
mu= bsxfun(@rdivide,transpose((transpose(P_lyi)*y)),sum(P_lyi))'
sumc=sum(P_lyi);
for l=1:L
sigma(:,:,l)=(bsxfun(@times, bsxfun(@minus,y,mu(l,:)) ,P_lyi(:,l)))' *(bsxfun(@minus,y,mu(l,:))) ;
sigma(:,:,l) = bsxfun(@rdivide,sigma(:,:,l),sumc(l));
end
However, densities of 4 observation from 783640 observation are zero in all components.
Your code seems very messy and difficult to check. At various places "l" (ell) is the first subscript and in others the second. Your use of transpose (especially in mu) seems incorrect. The core is P(t) [l|i] ... what is the sum over i of that measure? All the rest is algebra. I never use "I' (ell) as a subscript. Too easy to mix up with 1 (one) , I (capital eye) , | (vertical bar) , i (small i), ... you get the idea.
Assume we have data as P(ell given eye) ,,,,, "ell" is the column and "eye" is the row. Is your convention that vectors are row vectors and transpose is a column? Have you been 100% consistent with that? You can make a column vector of "1" (ones) and get the sum over "i" by PT x 1 --- or you can have a row vector of 1s and do 1 x P. You have to do all this carefully.
If this is all correct, you have to go back and see why is sum over i of P(t) [l|i] = 0.
- Badges
- Science topic