Teaching a Robot to Walk Using Reinforcement Learning
  Classical control techniques such as PID and LQR have been used effectively
in maintaining a system state, but these techniques become more difficult to
implement when the model dynamics increase in complexity and sensitivity. For
adaptive robotic locomotion tasks with several degrees of freedom, this task
becomes infeasible with classical control techniques. Instead, reinforcement
learning can train optimal walking policies with ease. We apply deep Q-learning
and augmented random search (ARS) to teach a simulated two-dimensional bipedal
robot how to walk using the OpenAI Gym BipedalWalker-v3 environment. Deep
Q-learning did not yield a high reward policy, often prematurely converging to
suboptimal local maxima likely due to the coarsely discretized action space.
ARS, however, resulted in a better trained robot, and produced an optimal
policy which officially "solves" the BipedalWalker-v3 problem. Various naive
policies, including a random policy, a manually encoded inch forward policy,
and a stay still policy, were used as benchmarks to evaluate the proficiency of
the learning algorithm results.
