An Energy-Saving Snake Locomotion Gait Policy Obtained Using Deep
  Reinforcement Learning
  Snake robots, comprised of sequentially connected joint actuators, have
recently gained increasing attention in the industrial field, like life
detection in narrow space. Such robots can navigate through the complex
environment via the cooperation of multiple motors located on the backbone.
However, controlling the robots in an unknown environment is challenging, and
conventional control strategies can be energy inefficient or even fail to
navigate to the destination. In this work, a snake locomotion gait policy is
developed via deep reinforcement learning (DRL) for energy-efficient control.
We apply proximal policy optimization (PPO) to each joint motor parameterized
by angular velocity and the DRL agent learns the standard serpenoid curve at
each timestep. The robot simulator and task environment are built upon
PyBullet. Comparing to conventional control strategies, the snake robots
controlled by the trained PPO agent can achieve faster movement and more
energy-efficient locomotion gait. This work demonstrates that DRL provides an
energy-efficient solution for robot control.
