Exploiting Action Impact Regularity and Exogenous State Variables for
  Offline Reinforcement Learning
  Offline reinforcement learning-learning a policy from a batch of data-is
known to be hard for general MDPs. In this work, we explore a restricted class
of MDPs to obtain guarantees for offline reinforcement learning. The key
property, which we call Action Impact Regularity (AIR), is that actions
primarily impact a part of the state (an endogenous component) with limited
impact on the remaining part of the state (an exogenous component). We propose
an algorithm that exploits the AIR property, and provide theoretical guarantees
for the outputted policy. Finally, we demonstrate that our algorithm
outperforms existing offline reinforcement learning algorithms across different
data collection policies in simulated and real world environments where the
regularity holds.
