A Tutorial on Distance Metric Learning: Mathematical Foundations,
  Algorithms, Experimental Analysis, Prospects and Challenges (with Appendices
  on Mathematical Background and Detailed Algorithms Explanation)
  Distance metric learning is a branch of machine learning that aims to learn
distances from the data, which enhances the performance of similarity-based
algorithms. This tutorial provides a theoretical background and foundations on
this topic and a comprehensive experimental analysis of the most-known
algorithms. We start by describing the distance metric learning problem and its
main mathematical foundations, divided into three main blocks: convex analysis,
matrix analysis and information theory. Then, we will describe a representative
set of the most popular distance metric learning methods used in
classification. All the algorithms studied in this paper will be evaluated with
exhaustive testing in order to analyze their capabilities in standard
classification problems, particularly considering dimensionality reduction and
kernelization. The results, verified by Bayesian statistical tests, highlight a
set of outstanding algorithms. Finally, we will discuss several potential
future prospects and challenges in this field. This tutorial will serve as a
starting point in the domain of distance metric learning from both a
theoretical and practical perspective.
