Building Reliable Explanations of Unreliable Neural Networks: Locally
  Smoothing Perspective of Model Interpretation
  We present a novel method for reliably explaining the predictions of neural
networks. We consider an explanation reliable if it identifies input features
relevant to the model output by considering the input and the neighboring data
points. Our method is built on top of the assumption of smooth landscape in a
loss function of the model prediction: locally consistent loss and gradient
profile. A theoretical analysis established in this study suggests that those
locally smooth model explanations are learned using a batch of noisy copies of
the input with the L1 regularization for a saliency map. Extensive experiments
support the analysis results, revealing that the proposed saliency maps
retrieve the original classes of adversarial examples crafted against both
naturally and adversarially trained models, significantly outperforming
previous methods. We further demonstrated that such good performance results
from the learning capability of this method to identify input features that are
truly relevant to the model output of the input and the neighboring data
points, fulfilling the requirements of a reliable explanation.
