Greetings,

When using the Expectation Maximization (EM) algorithm, what it is doing under the hood is comparing statistical mixtures. EM searches for k Gaussian distributions in the data, when k is explicitly defined, this is the special case where the results are the same as k-means, I believe this makes the need for sigma moot (somebody may be able to provide more rigorous insight on this). I am assuming you are using the unsupervised version (unknown number of latent classes) that starts with the unsupervised Naive Bayes, but EM also works with partially labeled data as well. In each case, the algorithm is guaranteed to converge because the log-likelihood is log-concave.

Best,

Tyler J Grear

Hi dear Tyler, thanks for your nice recreation. It's true we're using an unsupervised version, we don't know exactly the number of clusters for using the dataset.

But it is not clear for me the matching between k-Means and EM. Sure, as you said, EM tries to estimate parameters of the distribution when the k clusters are exactly known. Do you know if there is a particular value of the standard deviation (sigma) for the EM in which we can say that k-Means = EM (perhaps a special case of EM, I don't know!)?

Regards,

Anselme R. Affane M.

Hi

What you really mean is °Gaussian Mixture Models° not EM algorithm, this latter is a tool to estimate the former.

the two methods are inherently different, e.g. kMeans is deterministic VS GMM is probabilistic

In GMM one takes (covariance) into account, this is not done in kMeans

but Kmeans can be seen as a hard assign version of GMM

( Pattern Recognition and Machine Learning-Bishop)

K Means is a maximization problem similar as classifications, detections etc., that may be analyzed via EM or MLE etc. Do not be confuse about its equivalence.