Representing Deep Neural Networks Latent Space Geometries with Graphs
  Deep Learning (DL) has attracted a lot of attention for its ability to reach
state-of-the-art performance in many machine learning tasks. The core principle
of DL methods consists in training composite architectures in an end-to-end
fashion, where inputs are associated with outputs trained to optimize an
objective function. Because of their compositional nature, DL architectures
naturally exhibit several intermediate representations of the inputs, which
belong to so-called latent spaces. When treated individually, these
intermediate representations are most of the time unconstrained during the
learning process, as it is unclear which properties should be favored. However,
when processing a batch of inputs concurrently, the corresponding set of
intermediate representations exhibit relations (what we call a geometry) on
which desired properties can be sought. In this work, we show that it is
possible to introduce constraints on these latent geometries to address various
problems. In more details, we propose to represent geometries by constructing
similarity graphs from the intermediate representations obtained when
processing a batch of inputs. By constraining these Latent Geometry Graphs
(LGGs), we address the three following problems: i) Reproducing the behavior of
a teacher architecture is achieved by mimicking its geometry, ii) Designing
efficient embeddings for classification is achieved by targeting specific
geometries, and iii) Robustness to deviations on inputs is achieved via
enforcing smooth variation of geometry between consecutive latent spaces. Using
standard vision benchmarks, we demonstrate the ability of the proposed
geometry-based methods in solving the considered problems.
