Trajectory Tracking of Underactuated Sea Vessels With Uncertain
  Dynamics: An Integral Reinforcement Learning Approach
  Underactuated systems like sea vessels have degrees of motion that are
insufficiently matched by a set of independent actuation forces. In addition,
the underlying trajectory-tracking control problems grow in complexity in order
to decide the optimal rudder and thrust control signals. This enforces several
difficult-to-solve constraints that are associated with the error dynamical
equations using classical optimal tracking and adaptive control approaches. An
online machine learning mechanism based on integral reinforcement learning is
proposed to find a solution for a class of nonlinear tracking problems with
partial prior knowledge of the system dynamics. The actuation forces are
decided using innovative forms of temporal difference equations relevant to the
vessel's surge and angular velocities. The solution is implemented using an
online value iteration process which is realized by employing means of the
adaptive critics and gradient descent approaches. The adaptive learning
mechanism exhibited well-functioning and interactive features in react to
different desired reference-tracking scenarios.
