Robustness and Usefulness in AI Explanation Methods
  Explainability in machine learning has become incredibly important as machine
learning-powered systems become ubiquitous and both regulation and public
sentiment begin to demand an understanding of how these systems make decisions.
As a result, a number of explanation methods have begun to receive widespread
adoption. This work summarizes, compares, and contrasts three popular
explanation methods: LIME, SmoothGrad, and SHAP. We evaluate these methods with
respect to: robustness, in the sense of sample complexity and stability;
understandability, in the sense that provided explanations are consistent with
user expectations; and usability, in the sense that the explanations allow for
the model to be modified based on the output. This work concludes that current
explanation methods are insufficient; that putting faith in and adopting these
methods may actually be worse than simply not using them.
