What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation
  Framework for Explainability Methods
  A multitude of explainability methods and associated fidelity performance
metrics have been proposed to help better understand how modern AI systems make
decisions. However, much of the current work has remained theoretical --
without much consideration for the human end-user. In particular, it is not yet
known (1) how useful current explainability methods are in practice for more
real-world scenarios and (2) how well associated performance metrics accurately
predict how much knowledge individual explanations contribute to a human
end-user trying to understand the inner-workings of the system. To fill this
gap, we conducted psychophysics experiments at scale to evaluate the ability of
human participants to leverage representative attribution methods for
understanding the behavior of different image classifiers representing three
real-world scenarios: identifying bias in an AI system, characterizing the
visual strategy it uses for tasks that are too difficult for an untrained
non-expert human observer as well as understanding its failure cases. Our
results demonstrate that the degree to which individual attribution methods
help human participants better understand an AI system varied widely across
these scenarios. This suggests a critical need for the field to move past
quantitative improvements of current attribution methods towards the
development of complementary approaches that provide qualitatively different
sources of information to human end-users.
