How Good is your Explanation? Algorithmic Stability Measures to Assess
  the Quality of Explanations for Deep Neural Networks
  A plethora of methods have been proposed to explain how deep neural networks
reach their decisions but comparatively, little effort has been made to ensure
that the explanations produced by these methods are objectively relevant. While
several desirable properties for trustworthy explanations have been formulated,
objective measures have been harder to derive. Here, we propose two new
measures to evaluate explanations borrowed from the field of algorithmic
stability: mean generalizability MeGe and relative consistency ReCo. We conduct
extensive experiments on different network architectures, common explainability
methods, and several image datasets to demonstrate the benefits of the proposed
measures.In comparison to ours, popular fidelity measures are not sufficient to
guarantee trustworthy explanations.Finally, we found that 1-Lipschitz networks
produce explanations with higher MeGe and ReCo than common neural networks
while reaching similar accuracy. This suggests that 1-Lipschitz networks are a
relevant direction towards predictors that are more explainable and
trustworthy.
