Technical Report: Adaptive Control for Linearizable Systems Using
  On-Policy Reinforcement Learning
  This paper proposes a framework for adaptively learning a feedback
linearization-based tracking controller for an unknown system using
discrete-time model-free policy-gradient parameter update rules. The primary
advantage of the scheme over standard model-reference adaptive control
techniques is that it does not require the learned inverse model to be
invertible at all instances of time. This enables the use of general function
approximators to approximate the linearizing controller for the system without
having to worry about singularities. However, the discrete-time and stochastic
nature of these algorithms precludes the direct application of standard
machinery from the adaptive control literature to provide deterministic
stability proofs for the system. Nevertheless, we leverage these techniques
alongside tools from the stochastic approximation literature to demonstrate
that with high probability the tracking and parameter errors concentrate near
zero when a certain persistence of excitation condition is satisfied. A
simulated example of a double pendulum demonstrates the utility of the proposed
theory. 1
