Deep Interactive Reinforcement Learning for Path Following of Autonomous
  Underwater Vehicle
  Autonomous underwater vehicle (AUV) plays an increasingly important role in
ocean exploration. Existing AUVs are usually not fully autonomous and generally
limited to pre-planning or pre-programming tasks. Reinforcement learning (RL)
and deep reinforcement learning have been introduced into the AUV design and
research to improve its autonomy. However, these methods are still difficult to
apply directly to the actual AUV system because of the sparse rewards and low
learning efficiency. In this paper, we proposed a deep interactive
reinforcement learning method for path following of AUV by combining the
advantages of deep reinforcement learning and interactive RL. In addition,
since the human trainer cannot provide human rewards for AUV when it is running
in the ocean and AUV needs to adapt to a changing environment, we further
propose a deep reinforcement learning method that learns from both human
rewards and environmental rewards at the same time. We test our methods in two
path following tasks---straight line and sinusoids curve following of AUV by
simulating in the Gazebo platform. Our experimental results show that with our
proposed deep interactive RL method, AUV can converge faster than a DQN learner
from only environmental reward. Moreover, AUV learning with our deep RL from
both human and environmental rewards can also achieve a similar or even better
performance than that with the deep interactive RL method and can adapt to the
actual environment by further learning from environmental rewards.
