Learning Quadrupedal Locomotion over Challenging Terrain
  Some of the most challenging environments on our planet are accessible to
quadrupedal animals but remain out of reach for autonomous machines. Legged
locomotion can dramatically expand the operational domains of robotics.
However, conventional controllers for legged locomotion are based on elaborate
state machines that explicitly trigger the execution of motion primitives and
reflexes. These designs have escalated in complexity while falling short of the
generality and robustness of animal locomotion. Here we present a radically
robust controller for legged locomotion in challenging natural environments. We
present a novel solution to incorporating proprioceptive feedback in locomotion
control and demonstrate remarkable zero-shot generalization from simulation to
natural environments. The controller is trained by reinforcement learning in
simulation. It is based on a neural network that acts on a stream of
proprioceptive signals. The trained controller has taken two generations of
quadrupedal ANYmal robots to a variety of natural environments that are beyond
the reach of prior published work in legged locomotion. The controller retains
its robustness under conditions that have never been encountered during
training: deformable terrain such as mud and snow, dynamic footholds such as
rubble, and overground impediments such as thick vegetation and gushing water.
The presented work opens new frontiers for robotics and indicates that radical
robustness in natural environments can be achieved by training in much simpler
domains.
