Convergence Guarantees of Policy Optimization Methods for Markovian Jump
  Linear Systems
  Recently, policy optimization for control purposes has received renewed
attention due to the increasing interest in reinforcement learning. In this
paper, we investigate the convergence of policy optimization for quadratic
control of Markovian jump linear systems (MJLS). First, we study the
optimization landscape of direct policy optimization for MJLS, and, in
particular, show that despite the non-convexity of the resultant problem the
unique stationary point is the global optimal solution. Next, we prove that the
Gauss-Newton method and the natural policy gradient method converge to the
optimal state feedback controller for MJLS at a linear rate if initialized at a
controller which stabilizes the closed-loop dynamics in the mean square sense.
We propose a novel Lyapunov argument to fix a key stability issue in the
convergence proof. Finally, we present a numerical example to support our
theory. Our work brings new insights for understanding the performance of
policy learning methods on controlling unknown MJLS.
