Neural Network Attribution Methods for Problems in Geoscience: A Novel
  Synthetic Benchmark Dataset
  Despite the increasingly successful application of neural networks to many
problems in the geosciences, their complex and nonlinear structure makes the
interpretation of their predictions difficult, which limits model trust and
does not allow scientists to gain physical insights about the problem at hand.
Many different methods have been introduced in the emerging field of
eXplainable Artificial Intelligence (XAI), which aim at attributing the network
s prediction to specific features in the input domain. XAI methods are usually
assessed by using benchmark datasets (like MNIST or ImageNet for image
classification). However, an objective, theoretically derived ground truth for
the attribution is lacking for most of these datasets, making the assessment of
XAI in many cases subjective. Also, benchmark datasets specifically designed
for problems in geosciences are rare. Here, we provide a framework, based on
the use of additively separable functions, to generate attribution benchmark
datasets for regression problems for which the ground truth of the attribution
is known a priori. We generate a large benchmark dataset and train a fully
connected network to learn the underlying function that was used for
simulation. We then compare estimated heatmaps from different XAI methods to
the ground truth in order to identify examples where specific XAI methods
perform well or poorly. We believe that attribution benchmarks as the ones
introduced herein are of great importance for further application of neural
networks in the geosciences, and for more objective assessment and accurate
implementation of XAI methods, which will increase model trust and assist in
discovering new science.
