BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement
  Learning
  There has recently been a surge in research in batch Deep Reinforcement
Learning (DRL), which aims for learning a high-performing policy from a given
dataset without additional interactions with the environment. We propose a new
algorithm, Best-Action Imitation Learning (BAIL), which strives for both
simplicity and performance. BAIL learns a V function, uses the V function to
select actions it believes to be high-performing, and then uses those actions
to train a policy network using imitation learning. For the MuJoCo benchmark,
we provide a comprehensive experimental study of BAIL, comparing its
performance to four other batch Q-learning and imitation-learning schemes for a
large variety of batch datasets. Our experiments show that BAIL's performance
is much higher than the other schemes, and is also computationally much faster
than the batch Q-learning schemes.
