Online and Distributed learning of Gaussian mixture models by Bayesian
  Moment Matching
  The Gaussian mixture model is a classic technique for clustering and data
modeling that is used in numerous applications. With the rise of big data,
there is a need for parameter estimation techniques that can handle streaming
data and distribute the computation over several processors. While online
variants of the Expectation Maximization (EM) algorithm exist, their data
efficiency is reduced by a stochastic approximation of the E-step and it is not
clear how to distribute the computation over multiple processors. We propose a
Bayesian learning technique that lends itself naturally to online and
distributed computation. Since the Bayesian posterior is not tractable, we
project it onto a family of tractable distributions after each observation by
matching a set of sufficient moments. This Bayesian moment matching technique
compares favorably to online EM in terms of time and accuracy on a set of data
modeling benchmarks.
