Particle-based Energetic Variational Inference
  We introduce a new variational inference (VI) framework, called energetic
variational inference (EVI). It minimizes the VI objective function based on a
prescribed energy-dissipation law. Using the EVI framework, we can derive many
existing Particle-based Variational Inference (ParVI) methods, including the
popular Stein Variational Gradient Descent (SVGD) approach. More importantly,
many new ParVI schemes can be created under this framework. For illustration,
we propose a new particle-based EVI scheme, which performs the particle-based
approximation of the density first and then uses the approximated density in
the variational procedure, or "Approximation-then-Variation" for short. Thanks
to this order of approximation and variation, the new scheme can maintain the
variational structure at the particle level, and can significantly decrease the
KL-divergence in each iteration. Numerical experiments show the proposed method
outperforms some existing ParVI methods in terms of fidelity to the target
distribution.
