An Active Learning Framework for Efficient Robust Policy Search
  Robust Policy Search is the problem of learning policies that do not degrade
in performance when subject to unseen environment model parameters. It is
particularly relevant for transferring policies learned in a simulation
environment to the real world. Several existing approaches involve sampling
large batches of trajectories which reflect the differences in various possible
environments, and then selecting some subset of these to learn robust policies,
such as the ones that result in the worst performance. We propose an active
learning based framework, EffAcTS, to selectively choose model parameters for
this purpose so as to collect only as much data as necessary to select such a
subset. We apply this framework using Linear Bandits, and experimentally
validate the gains in sample efficiency and the performance of our approach on
standard continuous control tasks. We also present a Multi-Task Learning
perspective to the problem of Robust Policy Search, and draw connections from
our proposed framework to existing work on Multi-Task Learning.
