I am just attempting to classify some data with GMM-UBM MAP adaptation approach. Inspired of speaker recognition, Initially, I used same covariance threshold as it used in MFCC features based speaker recognition applications (0.1) but I was not able to adapt models. The features I am trying to classify, have a different range of values as compared to typical MFCC features, I started changing the covariance threhsold, and I was able to adapt the models. I want to know what is the significance of covariance threshold, and what is the optimal way to compute covariance threshold.
Not sure how to answer about the threshold, but the issue may come from the fact that your features are correlated, which makes GMM with diagonal covariance matrix (used by default in many tools) cannot properly model.
Check your classification setup. You should make sure you either use full covariance matrix, or decorrelate your features (as it is done in DCT step for MFCC)
@ Arseniy: I applied some feature transformation, and different features transformations work on different variance threshold. Which somehow indicates that variance threshold must be according to the variance in the features. There will always be a high co-relation in intra-class features and less in inter-class features. Or I am wrong ?
I mean high correlation of the features you use within a sample (a frame, or splice of frames). For example, it is not quite possible to train GMM with diagonal covariance matrix on filter banks. This was basically one of the reasons of widespread usage of cepstral features instead of spectral ones. As for tuning the thresholds, I could say use held out data set, but I am afraid to give a wrong advice to you. I would definitely start with checking which model you use and how features are distributed within a sample.
By the way, I forgot one more thing that may help having more stable parameter estimation. You should normalize your features to have zero mean and unit variance for sure
I agree with your first statement. Features within a sample are not correlated. I am able to train GMM-UBM and adapt models from UBM. The only issue is min-variance threshold if I use default Matlab threshold which is 0.1 then I am getting NAN but if the selected threshold is close to variance within samples then it works absolutely fine. But I think your last point is what I am missing more.
"By the way, I forgot one more thing that may help having more stable parameter estimation. You should normalize your features to have zero mean and unit variance for sure"
So to be clear on that, here you mean, "Normalized features and unit variance" within a sample. Did it get it correct? Thanks.
Not exactly, I mean each feature separately is normalized for all samples. This is sometimes referred to as standartization (like here http://scikit-learn.org/stable/modules/preprocessing.html#standardization-or-mean-removal-and-variance-scaling)
In speech recognition, a common approach is to normalize features per speaker. However, a global normalization per data set might also be useful in your case
- Badges
- Science topic