Learning Flat Latent Manifolds with VAEs
  Measuring the similarity between data points often requires domain knowledge,
which can in parts be compensated by relying on unsupervised methods such as
latent-variable models, where similarity/distance is estimated in a more
compact latent space. Prevalent is the use of the Euclidean metric, which has
the drawback of ignoring information about similarity of data stored in the
decoder, as captured by the framework of Riemannian geometry. We propose an
extension to the framework of variational auto-encoders allows learning flat
latent manifolds, where the Euclidean metric is a proxy for the similarity
between data points. This is achieved by defining the latent space as a
Riemannian manifold and by regularising the metric tensor to be a scaled
identity matrix. Additionally, we replace the compact prior typically used in
variational auto-encoders with a recently presented, more expressive
hierarchical one---and formulate the learning problem as a constrained
optimisation problem. We evaluate our method on a range of data-sets, including
a video-tracking benchmark, where the performance of our unsupervised approach
nears that of state-of-the-art supervised approaches, while retaining the
computational efficiency of straight-line-based approaches.
