Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain
  We propose an approach for inverse reinforcement learning from hetero-domain
which learns a reward function in the simulator, drawing on the demonstrations
from the real world. The intuition behind the method is that the reward
function should not only be oriented to imitate the experts, but should
encourage actions adjusted for the dynamics difference between the simulator
and the real world. To achieve this, the widely used GAN-inspired IRL method is
adopted, and its discriminator, recognizing policy-generating trajectories, is
modified with the quantification of dynamics difference. The training process
of the discriminator can yield the transferable reward function suitable for
simulator dynamics, which can be guaranteed by derivation. Effectively, our
method assigns higher rewards for demonstration trajectories which do not
exploit discrepancies between the two domains. With extensive experiments on
continuous control tasks, our method shows its effectiveness and demonstrates
its scalability to high-dimensional tasks.
