Clusters in Explanation Space: Inferring disease subtypes from model
  explanations
  Identification of disease subtypes and corresponding biomarkers can
substantially improve clinical diagnosis and treatment selection. Discovering
these subtypes in noisy, high dimensional biomedical data is often impossible
for humans and challenging for machines. We introduce a new approach to
facilitate the discovery of disease subtypes: Instead of analyzing the original
data, we train a diagnostic classifier (healthy vs. diseased) and extract
instance-wise explanations for the classifier's decisions. The distribution of
instances in the explanation space of our diagnostic classifier amplifies the
different reasons for belonging to the same class - resulting in a
representation that is uniquely useful for discovering latent subtypes. We
compare our ability to recover subtypes via cluster analysis on model
explanations to classical cluster analysis on the original data. In multiple
datasets with known ground-truth subclasses, most compellingly on UK Biobank
brain imaging data and transcriptome data from the Cancer Genome Atlas, we show
that cluster analysis on model explanations substantially outperforms the
classical approach. While we believe clustering in explanation space to be
particularly valuable for inferring disease subtypes, the method is more
general and applicable to any kind of sub-type identification.
