Goal-conditioned Batch Reinforcement Learning for Rotation Invariant
  Locomotion
  We propose a novel approach to learn goal-conditioned policies for locomotion
in a batch RL setting. The batch data is collected by a policy that is not
goal-conditioned. For the locomotion task, this translates to data collection
using a policy learnt by the agent for walking straight in one direction, and
using that data to learn a goal-conditioned policy that enables the agent to
walk in any direction. The data collection policy used should be invariant to
the direction the agent is facing i.e. regardless of its initial orientation,
the agent should take the same actions to walk forward. We exploit this
property to learn a goal-conditioned policy using two key ideas: (1) augmenting
data by generating trajectories with the same actions in different directions,
and (2) learning an encoder that enforces invariance between these rotated
trajectories with a Siamese framework. We show that our approach outperforms
existing RL algorithms on 3-D locomotion agents like Ant, Humanoid and
Minitaur.
