Assume we fit a GMM on a train set which contains normal samples. I'm wondering if we can use probability density function (which is a linear combination of component densities) in order to decide about new samples (test samples). In other words, our final decision about being normal or abnormal should be should be based on which probabilities: probability density function, Likelihood or posterior?
Hi Sajjad,
as Eric pointed out, the idea of using GMM for anomaly detection, OCC, etc. is not new.
Usually, one uses the estimated probability density p(x)=sum_i p(i)*p(x|i), i.e. the linear combination of cluster likelihoods you mentioned in your question above.
Conceptually, an anomaly is something that is unusual. There are two cases that fall under this category: (1) you least expect an event and it occurs, and (2) you are mostly certain about an event and it does not occur.
If A is an event and say the probability of the event A happening P(A) = 0.00001. This is pretty low probability. But if A happens it is an anomaly. In this case it was the prior probability of A.
If P(A) = 0.998 and event A did not happen, then it is an anomaly.
In terms of conditional probability, you may have to check the likelihood of an event occurrence and compare it with the actual event occurrence.
Low likelihood + event occurrence => anomaly
High likelihood + event absence => anomaly
Hope this gives some direction... I haven't used this idea myself :) Maybe someone who would have used these ideas in a real-world problem can give some insights here.
Thank you all my Dear friends for your replies
As Mr. Grim said I'm doing a One-Class classification. I've fitted a GMM on a class of data . Now I want to use the GMM to make decision about new samples. I have trained the GMM using EM algorithm and related parameters are determined. In this stage (testing phase) should I calculate test data probabilities ( p(x) = Sigma (w * p(x|j) ) and j indicated a Density Component ) and compare this pdf with a predefined Threshold?
Personally I think there are different approaches than one of them could be using pdf and one threshold in order to make final decisions.
Different methods of outlier detection may be better suited to different problems. Did you pick GMM for a particular reason, or just because that's what you're most familiar with?
Dear Williams
Thanks for your consideration
This is part of a bigger project and I have reasons for using GMM. But it's a little complicated and I can't explain the reasons here with comments.
I'm not personally familiar with GMM methods, but I know there are several references on them cited in this survey:
Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys (CSUR), 41(3), 15.
Hi Sajjad,
as Eric pointed out, the idea of using GMM for anomaly detection, OCC, etc. is not new.
Usually, one uses the estimated probability density p(x)=sum_i p(i)*p(x|i), i.e. the linear combination of cluster likelihoods you mentioned in your question above.
I think you can base your decision on the probability density values of the samples if you quantise them.
Hi
I am not using GMMs for event detection, or not in the same way than you. but I am using extensively GMMs in speaker recognition (usually large GMMs with diag. covariance matrices).
I suppose that you are using GMMs with a small number of components (take care huge GMMs have a very different behavior than small ones). In this situation, you are making an underlined hypothesis, where each event is associated with one gaussian component. If it is true, when a data is coming into the GMM, it should be related to an event, so to one of the component. The contribution of this component in the misture likelihood is very high compared to the others.
What I could suggest to you is to work on this characteristic in order to detect a new, unknown event. In this case, the contribution of the different likelihoods will be more balanced, with several "winners". It allows to build a nice "unknown event" detector, based on these contribution repartition, in a two classes approach (GMM with a data linked to a known event and GMM feed with a data coming from an unknown event)
hope I am clear...
Best
JF
- Badges
- Science topic