Explaining Latent Representations with a Corpus of Examples
  Modern machine learning models are complicated. Most of them rely on
convoluted latent representations of their input to issue a prediction. To
achieve greater transparency than a black-box that connects inputs to
predictions, it is necessary to gain a deeper understanding of these latent
representations. To that aim, we propose SimplEx: a user-centred method that
provides example-based explanations with reference to a freely selected set of
examples, called the corpus. SimplEx uses the corpus to improve the user's
understanding of the latent space with post-hoc explanations answering two
questions: (1) Which corpus examples explain the prediction issued for a given
test example? (2) What features of these corpus examples are relevant for the
model to relate them to the test example? SimplEx provides an answer by
reconstructing the test latent representation as a mixture of corpus latent
representations. Further, we propose a novel approach, the Integrated Jacobian,
that allows SimplEx to make explicit the contribution of each corpus feature in
the mixture. Through experiments on tasks ranging from mortality prediction to
image classification, we demonstrate that these decompositions are robust and
accurate. With illustrative use cases in medicine, we show that SimplEx
empowers the user by highlighting relevant patterns in the corpus that explain
model representations. Moreover, we demonstrate how the freedom in choosing the
corpus allows the user to have personalized explanations in terms of examples
that are meaningful for them.
