Online Algorithms and Policies Using Adaptive and Machine Learning
  Approaches
  This paper considers the problem of real-time control and learning in dynamic
systems subjected to parametric uncertainties and proposes a controller that
combines Adaptive Control (AC) in the inner loop and a Reinforcement Learning
(RL) based policy in the outer loop. Two classes of nonlinear dynamic systems
are considered, both of which are control-affine. The first class of dynamic
systems utilizes equilibrium points with expansion forms around these points
and employs a Lyapunov approach. The second class of nonlinear systems uses
contraction theory as the underlying framework. For both classes of systems,
the AC-RL controller is shown to lead to online policies that guarantee
stability, and leverage accelerated convergence properties using a high-order
tuner. Additionally, for the second class of systems, the AC-RL controller is
shown to lead to parameter learning with persistent excitation. Numerical
validations of all algorithms are carried out using a quadrotor landing task on
a moving platform and other academic examples. All results clearly point out
the advantage of the proposed integrative AC-RL approach.
