REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy
  Transfer
  A popular paradigm in robotic learning is to train a policy from scratch for
every new robot. This is not only inefficient but also often impractical for
complex robots. In this work, we consider the problem of transferring a policy
across two different robots with significantly different parameters such as
kinematics and morphology. Existing approaches that train a new policy by
matching the action or state transition distribution, including imitation
learning methods, fail due to optimal action and/or state distribution being
mismatched in different robots. In this paper, we propose a novel method named
$REvolveR$ of using continuous evolutionary models for robotic policy transfer
implemented in a physics simulator. We interpolate between the source robot and
the target robot by finding a continuous evolutionary change of robot
parameters. An expert policy on the source robot is transferred through
training on a sequence of intermediate robots that gradually evolve into the
target robot. Experiments show that the proposed continuous evolutionary model
can effectively transfer the policy across robots and achieve superior sample
efficiency on new robots using a physics simulator. The proposed method is
especially advantageous in sparse reward settings where exploration can be
significantly reduced.
