On the Optimization Landscape of Maximum Mean Discrepancy
  Generative models have been successfully used for generating realistic
signals. Because the likelihood function is typically intractable in most of
these models, the common practice is to use "implicit" models that avoid
likelihood calculation. However, it is hard to obtain theoretical guarantees
for such models. In particular, it is not understood when they can globally
optimize their non-convex objectives. Here we provide such an analysis for the
case of Maximum Mean Discrepancy (MMD) learning of generative models. We prove
several optimality results, including for a Gaussian distribution with low rank
covariance (where likelihood is inapplicable) and a mixture of Gaussians. Our
analysis shows that that the MMD optimization landscape is benign in these
cases, and therefore gradient based methods will globally minimize the MMD
objective.
