Unified Perspective on Probability Divergence via Maximum Likelihood
  Density Ratio Estimation: Bridging KL-Divergence and Integral Probability
  Metrics
  This paper provides a unified perspective for the Kullback-Leibler
(KL)-divergence and the integral probability metrics (IPMs) from the
perspective of maximum likelihood density-ratio estimation (DRE). Both the
KL-divergence and the IPMs are widely used in various fields in applications
such as generative modeling. However, a unified understanding of these concepts
has still been unexplored. In this paper, we show that the KL-divergence and
the IPMs can be represented as maximal likelihoods differing only by sampling
schemes, and use this result to derive a unified form of the IPMs and a relaxed
estimation method. To develop the estimation problem, we construct an
unconstrained maximum likelihood estimator to perform DRE with a stratified
sampling scheme. We further propose a novel class of probability divergences,
called the Density Ratio Metrics (DRMs), that interpolates the KL-divergence
and the IPMs. In addition to these findings, we also introduce some
applications of the DRMs, such as DRE and generative adversarial networks. In
experiments, we validate the effectiveness of our proposed methods.
