Transparency and Explanation in Deep Reinforcement Learning Neural
  Networks
  Autonomous AI systems will be entering human society in the near future to
provide services and work alongside humans. For those systems to be accepted
and trusted, the users should be able to understand the reasoning process of
the system, i.e. the system should be transparent. System transparency enables
humans to form coherent explanations of the system's decisions and actions.
Transparency is important not only for user trust, but also for software
debugging and certification. In recent years, Deep Neural Networks have made
great advances in multiple application areas. However, deep neural networks are
opaque. In this paper, we report on work in transparency in Deep Reinforcement
Learning Networks (DRLN). Such networks have been extremely successful in
accurately learning action control in image input domains, such as Atari games.
In this paper, we propose a novel and general method that (a) incorporates
explicit object recognition processing into deep reinforcement learning models,
(b) forms the basis for the development of "object saliency maps", to provide
visualization of internal states of DRLNs, thus enabling the formation of
explanations and (c) can be incorporated in any existing deep reinforcement
learning framework. We present computational results and human experiments to
evaluate our approach.
