Guided Curriculum Learning for Walking Over Complex Terrain
  Reliable bipedal walking over complex terrain is a challenging problem, using
a curriculum can help learning. Curriculum learning is the idea of starting
with an achievable version of a task and increasing the difficulty as a success
criteria is met. We propose a 3-stage curriculum to train Deep Reinforcement
Learning policies for bipedal walking over various challenging terrains. In the
first stage, the agent starts on an easy terrain and the terrain difficulty is
gradually increased, while forces derived from a target policy are applied to
the robot joints and the base. In the second stage, the guiding forces are
gradually reduced to zero. Finally, in the third stage, random perturbations
with increasing magnitude are applied to the robot base, so the robustness of
the policies are improved. In simulation experiments, we show that our approach
is effective in learning walking policies, separate from each other, for five
terrain types: flat, hurdles, gaps, stairs, and steps. Moreover, we demonstrate
that in the absence of human demonstrations, a simple hand designed walking
trajectory is a sufficient prior to learn to traverse complex terrain types. In
ablation studies, we show that taking out any one of the three stages of the
curriculum degrades the learning performance.
