In the Expectation Maximum (EM) algorithm, does the expectation step usually adopt a linear optimation model to create a function for the expectation of the log-likelihood evaluated using the current estimate for the parameters? That is to say, does the E-step always use a set of linear equations to obtain the estimate of the latent variables? Thanks.
The following extends linear constraints in loglinear modeling to convex constraints.
Winkler, W. E. (1990c), “On Dykstra’s Iterative Fitting Procedure,” Annals of Probability, 18, 1410-1415.
The following generalizes the ECM algorithm of Meng and Rubin *Biometrika 1993) from linear to convex constraints. You need to be careful during both the E-Step and the M-Step to assure that the convex constraints are satisfied.
Winkler, W. E. (1993b), "Improved Decision Rules in the Fellegi-Sunter Model of Record Linkage," Proceedings of the Section on Survey Research Methods, American Statistical Association, 274-279, also http://www.census.gov/srd/papers/pdf/rr93-12.pdf .
In the following two papers, I use carefully constructed convex constraints to put upper and lower bounds of sets of cells in a contingency table. The model with the convex constraints has almost the same Kullback-Leibler distance as the one with linear constraints (meaning basic analytic properties are preserved).
Winkler, W. E. (2008b), “General Methods and Algorithms for Imputing Discrete Data under a Variety of Constraints,” http://www.census.gov/srd/papers/pdf/rrs2008-08.pdf .
Winkler, W. E. (2010), General Discrete-data Modeling Methods for Creating Synthetic Data with Reduced Re-identification Risk that Preserve Analytic Properties, http://www.census.gov/srd/papers/pdf/rrs2010-02.pdf .
It depends on the specific problems. For EM algorithm, in E step, we are required to calculate the expected probability of the missing data given the observed data and the parameters. When the observed data and parameters are available, the relationship between the missing data and them are not always linear.
https://people.eecs.berkeley.edu/~pabbeel/cs287-fa13/slides/Likelihood_EM_HMM_Kalman.pdf
http://www.cse.psu.edu/~rtc12/CSE586Spring2010/papers/prmlMixturesEM.pdf
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Algorithms