Convergence of the EM Algorithm for Gaussian Mixtures with Unbalanced
  Mixing Coefficients
  The speed of convergence of the Expectation Maximization (EM) algorithm for
Gaussian mixture model fitting is known to be dependent on the amount of
overlap among the mixture components. In this paper, we study the impact of
mixing coefficients on the convergence of EM. We show that when the mixture
components exhibit some overlap, the convergence of EM becomes slower as the
dynamic range among the mixing coefficients increases. We propose a
deterministic anti-annealing algorithm, that significantly improves the speed
of convergence of EM for such mixtures with unbalanced mixing coefficients. The
proposed algorithm is compared against other standard optimization techniques
like BFGS, Conjugate Gradient, and the traditional EM algorithm. Finally, we
propose a similar deterministic anti-annealing based algorithm for the
Dirichlet process mixture model and demonstrate its advantages over the
conventional variational Bayesian approach.
