I started recently to read about HMM topic, particularly Continuous HMM. Generally, people model the emission probability distribution of each state , in Continuous HMM, using a multivariate normal distribution or even a mixture gaussian distribution. But why exactly this kind of distribution?

In [1] they said that's commune, in [2] it's tractable and in [3] they talked about some interesting properties of the exponential distribution family. In voice/handwriting recognition they often use a mixture of gaussians. In [4] they give a great arguments about why the normal distribution/mixture of gaussians may solve the problem of estimating the HMM parameters (emission parameters), and the way to re-estimate its parameters using Baum-Welch algorithm ( a complete framework to estimate the HMM parameters).

The emission distributions are basic in HMM modeling, and using a mixture of gaussian for each state in high dimension space needs a huge parameters to estimate. So the questions are:

(1) Can we use other kind of distributions/models instead of gaussians mixture (generative model), to answer the question: what's the probability that an observation x belongs to a state s?

(2) Can we use a discriminative model to replace all the distributions in the same time, e.g: a multi-classifier. that it can give the probability that a vector x belongs to each class? (for example: a neural network, or multi-SVM classifier ...).

The main problem of these new methods is to find a way to update their parameters using Baum-Welch algorithm during the training process. This problem may complicate seriously the use of other distributions family/classifier instead of the guassian mixture.

Any comments are welcome.

[1] C. Bishop. "Pattern Recognition and Machine Learning", Springer, 2006.

[2] K.Murphy "Machine Learning, a probabilistic perspective", The MIT press, 2012.

[3] J. Norris. "Markov Chains". 1997.

[4] J. Gauvain, C. Lee. "Bayesian Learning of Gaussian Mixture Densities for Hidden Markov Models". Proc. DARPA Speech and Natural Language Workshop. 1991. P, 272-277.