Deep Divergence Learning
  Classical linear metric learning methods have recently been extended along
two distinct lines: deep metric learning methods for learning embeddings of the
data using neural networks, and Bregman divergence learning approaches for
extending learning Euclidean distances to more general divergence measures such
as divergences over distributions. In this paper, we introduce deep Bregman
divergences, which are based on learning and parameterizing functional Bregman
divergences using neural networks, and which unify and extend these existing
lines of work. We show in particular how deep metric learning formulations,
kernel metric learning, Mahalanobis metric learning, and moment-matching
functions for comparing distributions arise as special cases of these
divergences in the symmetric setting. We then describe a deep learning
framework for learning general functional Bregman divergences, and show in
experiments that this method yields superior performance on benchmark datasets
as compared to existing deep metric learning approaches. We also discuss novel
applications, including a semi-supervised distributional clustering problem,
and a new loss function for unsupervised data generation.
