"Why did you do that?": Explaining black box models with Inductive
  Synthesis
  By their nature, the composition of black box models is opaque. This makes
the ability to generate explanations for the response to stimuli challenging.
The importance of explaining black box models has become increasingly important
given the prevalence of AI and ML systems and the need to build legal and
regulatory frameworks around them. Such explanations can also increase trust in
these uncertain systems. In our paper we present RICE, a method for generating
explanations of the behaviour of black box models by (1) probing a model to
extract model output examples using sensitivity analysis; (2) applying
CNPInduce, a method for inductive logic program synthesis, to generate logic
programs based on critical input-output pairs; and (3) interpreting the target
program as a human-readable explanation. We demonstrate the application of our
method by generating explanations of an artificial neural network trained to
follow simple traffic rules in a hypothetical self-driving car simulation. We
conclude with a discussion on the scalability and usability of our approach and
its potential applications to explanation-critical scenarios.
