Feedback Linearization for Unknown Systems via Reinforcement Learning
  We present a novel approach to control design for nonlinear systems which
leverages model-free policy optimization techniques to learn a linearizing
controller for a physical plant with unknown dynamics. Feedback linearization
is a technique from nonlinear control which renders the input-output dynamics
of a nonlinear plant \emph{linear} under application of an appropriate feedback
controller. Once a linearizing controller has been constructed, desired output
trajectories for the nonlinear plant can be tracked using a variety of linear
control techniques. However, the calculation of a linearizing controller
requires a precise dynamics model for the system. As a result, model-based
approaches for learning exact linearizing controllers generally require a
simple, highly structured model of the system with easily identifiable
parameters. In contrast, the model-free approach presented in this paper is
able to approximate the linearizing controller for the plant using general
function approximation architectures. Specifically, we formulate a
continuous-time optimization problem over the parameters of a learned
linearizing controller whose optima are the set of parameters which best
linearize the plant. We derive conditions under which the learning problem is
(strongly) convex and provide guarantees which ensure the true linearizing
controller for the plant is recovered. We then discuss how model-free policy
optimization algorithms can be used to solve a discrete-time approximation to
the problem using data collected from the real-world plant. The utility of the
framework is demonstrated in simulation and on a real-world robotic platform.
