An Adaptive EM Accelerator for Unsupervised Learning of Gaussian Mixture
  Models
  We propose an Anderson Acceleration (AA) scheme for the adaptive
Expectation-Maximization (EM) algorithm for unsupervised learning a finite
mixture model from multivariate data (Figueiredo and Jain 2002). The proposed
algorithm is able to determine the optimal number of mixture components
autonomously, and converges to the optimal solution much faster than its
non-accelerated version. The success of the AA-based algorithm stems from
several developments rather than a single breakthrough (and without these, our
tests demonstrate that AA fails catastrophically). To begin, we ensure the
monotonicity of the likelihood function (a the key feature of the standard EM
algorithm) with a recently proposed monotonicity-control algorithm (Henderson
and Varahdan 2019), enhanced by a novel monotonicity test with little overhead.
We propose nimble strategies for AA to preserve the positive definiteness of
the Gaussian weights and covariance matrices strictly, and to conserve up to
the second moments of the observed data set exactly. Finally, we employ a
K-means clustering algorithm using the gap statistic to avoid excessively
overestimating the initial number of components, thereby maximizing
performance. We demonstrate the accuracy and efficiency of the algorithm with
several synthetic data sets that are mixtures of Gaussians distributions of
known number of components, as well as data sets generated from
particle-in-cell simulations. Our numerical results demonstrate speed-ups with
respect to non-accelerated EM of up to 60X when the exact number of mixture
components is known, and between a few and more than an order of magnitude with
component adaptivity.
