Desiderata for Representation Learning: A Causal Perspective
  Representation learning constructs low-dimensional representations to
summarize essential features of high-dimensional data. This learning problem is
often approached by describing various desiderata associated with learned
representations; e.g., that they be non-spurious, efficient, or disentangled.
It can be challenging, however, to turn these intuitive desiderata into formal
criteria that can be measured and enhanced based on observed data. In this
paper, we take a causal perspective on representation learning, formalizing
non-spuriousness and efficiency (in supervised representation learning) and
disentanglement (in unsupervised representation learning) using counterfactual
quantities and observable consequences of causal assertions. This yields
computable metrics that can be used to assess the degree to which
representations satisfy the desiderata of interest and learn non-spurious and
disentangled representations from single observational datasets.
