Imitating, Fast and Slow: Robust learning from demonstrations via
  decision-time planning
  The goal of imitation learning is to mimic expert behavior from
demonstrations, without access to an explicit reward signal. A popular class of
approach infers the (unknown) reward function via inverse reinforcement
learning (IRL) followed by maximizing this reward function via reinforcement
learning (RL). The policies learned via these approaches are however very
brittle in practice and deteriorate quickly even with small test-time
perturbations due to compounding errors. We propose Imitation with Planning at
Test-time (IMPLANT), a new meta-algorithm for imitation learning that utilizes
decision-time planning to correct for compounding errors of any base imitation
policy. In contrast to existing approaches, we retain both the imitation policy
and the rewards model at decision-time, thereby benefiting from the learning
signal of the two components. Empirically, we demonstrate that IMPLANT
significantly outperforms benchmark imitation learning approaches on standard
control environments and excels at zero-shot generalization when subject to
challenging perturbations in test-time dynamics.
