Explaining Conditions for Reinforcement Learning Behaviors from Real and
  Imagined Data
  The deployment of reinforcement learning (RL) in the real world comes with
challenges in calibrating user trust and expectations. As a step toward
developing RL systems that are able to communicate their competencies, we
present a method of generating human-interpretable abstract behavior models
that identify the experiential conditions leading to different task execution
strategies and outcomes. Our approach consists of extracting experiential
features from state representations, abstracting strategy descriptors from
trajectories, and training an interpretable decision tree that identifies the
conditions most predictive of different RL behaviors. We demonstrate our method
on trajectory data generated from interactions with the environment and on
imagined trajectory data that comes from a trained probabilistic world model in
a model-based RL setting.
