Hierarchical Reinforcement Learning for Quadruped Locomotion
  Legged locomotion is a challenging task for learning algorithms, especially
when the task requires a diverse set of primitive behaviors. To solve these
problems, we introduce a hierarchical framework to automatically decompose
complex locomotion tasks. A high-level policy issues commands in a latent space
and also selects for how long the low-level policy will execute the latent
command. Concurrently, the low-level policy uses the latent command and only
the robot's on-board sensors to control the robot's actuators. Our approach
allows the high-level policy to run at a lower frequency than the low-level
one. We test our framework on a path-following task for a dynamic quadruped
robot and we show that steering behaviors automatically emerge in the latent
command space as low-level skills are needed for this task. We then show
efficient adaptation of the trained policy to a different task by transfer of
the trained low-level policy. Finally, we validate the policies on a real
quadruped robot. To the best of our knowledge, this is the first application of
end-to-end hierarchical learning to a real robotic locomotion task.
