Fast approximations of the Jeffreys divergence between univariate
  Gaussian mixture models via exponential polynomial densities
  The Jeffreys divergence is a renown symmetrization of the oriented
Kullback-Leibler divergence broadly used in information sciences. Since the
Jeffreys divergence between Gaussian mixture models is not available in
closed-form, various techniques with pros and cons have been proposed in the
literature to either estimate, approximate, or lower and upper bound this
divergence. In this paper, we propose a simple yet fast heuristic to
approximate the Jeffreys divergence between two univariate Gaussian mixtures
with arbitrary number of components. Our heuristic relies on converting the
mixtures into pairs of dually parameterized probability densities belonging to
an exponential family. In particular, we consider the versatile polynomial
exponential family densities, and design a divergence to measure in closed-form
the goodness of fit between a Gaussian mixture and its polynomial exponential
density approximation. This goodness-of-fit divergence is a generalization of
the Hyv\"arinen divergence used to estimate models with computationally
intractable normalizers. It allows us to perform model selection by choosing
the orders of the polynomial exponential densities used to approximate the
mixtures. We demonstrate experimentally that our heuristic to approximate the
Jeffreys divergence improves by several orders of magnitude the computational
time of stochastic Monte Carlo estimations while approximating reasonably well
the Jeffreys divergence, specially when the mixtures have a very small number
of modes. Besides, our mixture-to-exponential family conversion techniques may
prove useful in other settings.
