Spectral, Probabilistic, and Deep Metric Learning: Tutorial and Survey
  This is a tutorial and survey paper on metric learning. Algorithms are
divided into spectral, probabilistic, and deep metric learning. We first start
with the definition of distance metric, Mahalanobis distance, and generalized
Mahalanobis distance. In spectral methods, we start with methods using scatters
of data, including the first spectral metric learning, relevant methods to
Fisher discriminant analysis, Relevant Component Analysis (RCA), Discriminant
Component Analysis (DCA), and the Fisher-HSIC method. Then, large-margin metric
learning, imbalanced metric learning, locally linear metric adaptation, and
adversarial metric learning are covered. We also explain several kernel
spectral methods for metric learning in the feature space. We also introduce
geometric metric learning methods on the Riemannian manifolds. In probabilistic
methods, we start with collapsing classes in both input and feature spaces and
then explain the neighborhood component analysis methods, Bayesian metric
learning, information theoretic methods, and empirical risk minimization in
metric learning. In deep learning methods, we first introduce reconstruction
autoencoders and supervised loss functions for metric learning. Then, Siamese
networks and its various loss functions, triplet mining, and triplet sampling
are explained. Deep discriminant analysis methods, based on Fisher discriminant
analysis, are also reviewed. Finally, we introduce multi-modal deep metric
learning, geometric metric learning by neural networks, and few-shot metric
learning.
