An Imitation from Observation Approach to Transfer Learning with
  Dynamics Mismatch
  We examine the problem of transferring a policy learned in a source
environment to a target environment with different dynamics, particularly in
the case where it is critical to reduce the amount of interaction with the
target environment during learning. This problem is particularly important in
sim-to-real transfer because simulators inevitably model real-world dynamics
imperfectly. In this paper, we show that one existing solution to this transfer
problem - grounded action transformation - is closely related to the problem of
imitation from observation (IfO): learning behaviors that mimic the
observations of behavior demonstrations. After establishing this relationship,
we hypothesize that recent state-of-the-art approaches from the IfO literature
can be effectively repurposed for grounded transfer learning.To validate our
hypothesis we derive a new algorithm - generative adversarial reinforced action
transformation (GARAT) - based on adversarial imitation from observation
techniques. We run experiments in several domains with mismatched dynamics, and
find that agents trained with GARAT achieve higher returns in the target
environment compared to existing black-box transfer methods
