Active Imitation Learning via Reduction to I.I.D. Active Learning
  In standard passive imitation learning, the goal is to learn a target policy
by passively observing full execution trajectories of it. Unfortunately,
generating such trajectories can require substantial expert effort and be
impractical in some cases. In this paper, we consider active imitation learning
with the goal of reducing this effort by querying the expert about the desired
action at individual states, which are selected based on answers to past
queries and the learner's interactions with an environment simulator. We
introduce a new approach based on reducing active imitation learning to i.i.d.
active learning, which can leverage progress in the i.i.d. setting. Our first
contribution, is to analyze reductions for both non-stationary and stationary
policies, showing that the label complexity (number of queries) of active
imitation learning can be substantially less than passive learning. Our second
contribution, is to introduce a practical algorithm inspired by the reductions,
which is shown to be highly effective in four test domains compared to a number
of alternatives.
