Focus! Rating XAI Methods and Finding Biases
  AI explainability improves the transparency of models, making them more
trustworthy. Such goals are motivated by the emergence of deep learning models,
which are obscure by nature; even in the domain of images, where deep learning
has succeeded the most, explainability is still poorly assessed. In the field
of image recognition many feature attribution methods have been proposed with
the purpose of explaining a model's behavior using visual cues. However, no
metrics have been established so far to assess and select these methods
objectively. In this paper we propose a consistent evaluation score for feature
attribution methods -- the Focus -- designed to quantify their coherency to the
task. While most previous work adds out-of-distribution noise to samples, we
introduce a methodology to add noise from within the distribution. This is done
through mosaics of instances from different classes, and the explanations these
generate. On those, we compute a visual pseudo-precision metric, Focus. First,
we show the robustness of the approach through a set of randomization
experiments. Then we use Focus to compare six popular explainability techniques
across several CNN architectures and classification datasets. Our results find
some methods to be consistently reliable (LRP, GradCAM), while others produce
class-agnostic explanations (SmoothGrad, IG). Finally we introduce another
application of Focus, using it for the identification and characterization of
biases found in models. This empowers bias-management tools, in another small
step towards trustworthy AI.
