Explainable Deterministic MDPs
  We present a method for a certain class of Markov Decision Processes (MDPs)
that can relate the optimal policy back to one or more reward sources in the
environment. For a given initial state, without fully computing the value
function, q-value function, or the optimal policy the algorithm can determine
which rewards will and will not be collected, whether a given reward will be
collected only once or continuously, and which local maximum within the value
function the initial state will ultimately lead to. We demonstrate that the
method can be used to map the state space to identify regions that are
dominated by one reward source and can fully analyze the state space to explain
all actions. We provide a mathematical framework to show how all of this is
possible without first computing the optimal policy or value function.
