Robust Batch Policy Learning in Markov Decision Processes
  We study the offline data-driven sequential decision making problem in the
framework of Markov decision process (MDP). In order to enhance the
generalizability and adaptivity of the learned policy, we propose to evaluate
each policy by a set of the average rewards with respect to distributions
centered at the policy induced stationary distribution. Given a pre-collected
dataset of multiple trajectories generated by some behavior policy, our goal is
to learn a robust policy in a pre-specified policy class that can maximize the
smallest value of this set. Leveraging the theory of semi-parametric
statistics, we develop a statistically efficient policy learning method for
estimating the de ned robust optimal policy. A rate-optimal regret bound up to
a logarithmic factor is established in terms of total decision points in the
dataset.
