Fair Marriage Principle and Initialization Map for the EM Algorithm
  The popular convergence theory of the EM algorithm explains that the observed
incomplete data log-likelihood L and the complete data log-likelihood Q are
positively correlated, and we can maximize L by maximizing Q. The Deterministic
Annealing EM (DAEM) algorithm was hence proposed for avoiding locally maximal
Q. This paper provides different conclusions: 1) The popular convergence theory
is wrong; 2) The locally maximal Q can affect the convergent speed, but cannot
block the global convergence; 3) Like marriage competition, unfair competition
between two components may vastly decrease the globally convergent speed; 4)
Local convergence exists because the sample is too small, and unfair
competition exists; 5) An improved EM algorithm, called the Channel Matching
(CM) EM algorithm, can accelerate the global convergence. This paper provides
an initialization map with two means as two axes for the example of a binary
Gaussian mixture studied by the authors of DAEM algorithm. This map can tell
how fast the convergent speeds are for different initial means and why points
in some areas are not suitable as initial points. A two-dimensional example
indicates that the big sample or the fair initialization can avoid global
convergence. For more complicated mixture models, we need further study to
convert the fair marriage principle to specific methods for the
initializations.
