Explainable Reinforcement Learning Through a Causal Lens
  Prevalent theories in cognitive science propose that humans understand and
represent the knowledge of the world through causal relationships. In making
sense of the world, we build causal models in our mind to encode cause-effect
relations of events and use these to explain why new events happen. In this
paper, we use causal models to derive causal explanations of behaviour of
reinforcement learning agents. We present an approach that learns a structural
causal model during reinforcement learning and encodes causal relationships
between variables of interest. This model is then used to generate explanations
of behaviour based on counterfactual analysis of the causal model. We report on
a study with 120 participants who observe agents playing a real-time strategy
game (Starcraft II) and then receive explanations of the agents' behaviour. We
investigated: 1) participants' understanding gained by explanations through
task prediction; 2) explanation satisfaction and 3) trust. Our results show
that causal model explanations perform better on these measures compared to two
other baseline explanation models.
