Robust Recovery Controller for a Quadrupedal Robot using Deep
  Reinforcement Learning
  The ability to recover from a fall is an essential feature for a legged robot
to navigate in challenging environments robustly. Until today, there has been
very little progress on this topic. Current solutions mostly build upon
(heuristically) predefined trajectories, resulting in unnatural behaviors and
requiring considerable effort in engineering system-specific components. In
this paper, we present an approach based on model-free Deep Reinforcement
Learning (RL) to control recovery maneuvers of quadrupedal robots using a
hierarchical behavior-based controller. The controller consists of four neural
network policies including three behaviors and one behavior selector to
coordinate them. Each of them is trained individually in simulation and
deployed directly on a real system. We experimentally validate our approach on
the quadrupedal robot ANYmal, which is a dog-sized quadrupedal system with 12
degrees of freedom. With our method, ANYmal manifests dynamic and reactive
recovery behaviors to recover from an arbitrary fall configuration within less
than 5 seconds. We tested the recovery maneuver more than 100 times, and the
success rate was higher than 97 %.
