Batch Active Preference-Based Learning of Reward Functions
  Data generation and labeling are usually an expensive part of learning for
robotics. While active learning methods are commonly used to tackle the former
problem, preference-based learning is a concept that attempts to solve the
latter by querying users with preference questions. In this paper, we will
develop a new algorithm, batch active preference-based learning, that enables
efficient learning of reward functions using as few data samples as possible
while still having short query generation times. We introduce several
approximations to the batch active learning problem, and provide theoretical
guarantees for the convergence of our algorithms. Finally, we present our
experimental results for a variety of robotics tasks in simulation. Our results
suggest that our batch active learning algorithm requires only a few queries
that are computed in a short amount of time. We then showcase our algorithm in
a study to learn human users' preferences.
