Improving performance of deep learning models with axiomatic attribution
  priors and expected gradients
  Recent research has demonstrated that feature attribution methods for deep
networks can themselves be incorporated into training; these attribution priors
optimize for a model whose attributions have certain desirable properties --
most frequently, that particular features are important or unimportant. These
attribution priors are often based on attribution methods that are not
guaranteed to satisfy desirable interpretability axioms, such as completeness
and implementation invariance. Here, we introduce attribution priors to
optimize for higher-level properties of explanations, such as smoothness and
sparsity, enabled by a fast new attribution method formulation called expected
gradients that satisfies many important interpretability axioms. This improves
model performance on many real-world tasks where previous attribution priors
fail. Our experiments show that the gains from combining higher-level
attribution priors with expected gradients attributions are consistent across
image, gene expression, and health care data sets. We believe this work
motivates and provides the necessary tools to support the widespread adoption
of axiomatic attribution priors in many areas of applied machine learning. The
implementations and our results have been made freely available to academic
communities.
