Learning Extremal Representations with Deep Archetypal Analysis
  Archetypes are typical population representatives in an extremal sense, where
typicality is understood as the most extreme manifestation of a trait or
feature. In linear feature space, archetypes approximate the data convex hull
allowing all data points to be expressed as convex mixtures of archetypes.
However, it might not always be possible to identify meaningful archetypes in a
given feature space. Learning an appropriate feature space and identifying
suitable archetypes simultaneously addresses this problem. This paper
introduces a generative formulation of the linear archetype model,
parameterized by neural networks. By introducing the distance-dependent
archetype loss, the linear archetype model can be integrated into the latent
space of a variational autoencoder, and an optimal representation with respect
to the unknown archetypes can be learned end-to-end. The reformulation of
linear Archetypal Analysis as deep variational information bottleneck, allows
the incorporation of arbitrarily complex side information during training.
Furthermore, an alternative prior, based on a modified Dirichlet distribution,
is proposed. The real-world applicability of the proposed method is
demonstrated by exploring archetypes of female facial expressions while using
multi-rater based emotion scores of these expressions as side information. A
second application illustrates the exploration of the chemical space of small
organic molecules. In this experiment, it is demonstrated that exchanging the
side information but keeping the same set of molecules, e. g. using as side
information the heat capacity of each molecule instead of the band gap energy,
will result in the identification of different archetypes. As an application,
these learned representations of chemical space might reveal distinct starting
points for de novo molecular design.
