Gaussian Mixture Reduction with Composite Transportation Divergence
  Gaussian mixture reduction (GMR) is the problem of approximating a high order
Gaussian mixture by one with lower order. It is widely used in density
estimation, recursive tracking in hidden Markov model, and belief propagation.
In this work, we show that the GMR can be formulated as an optimization problem
which minimizes the composite transportation divergence (CTD) between two
mixtures. The optimization problem can be solved by an easy-to-implement
Majorization-Minimization (MM) algorithm. We show that the MM algorithm
converges under general conditions. One popular computationally efficient
approach for GMR is the clustering based iterative algorithms. However, these
algorithms lack a theoretical guarantee whether they converge or attain some
optimality targets when they do. We show that existing clustering-based
algorithms are special cases of our MM algorithm can their theoretical
properties are therefore established. We further show the performance of the
clustering-based algorithms can be further improved by choosing various cost
function in the CTD. Numerical experiments are conducted to illustrate the
effectiveness of our proposed extension.
