Learning robust control for LQR systems with multiplicative noise via
  policy gradient
  The linear quadratic regulator (LQR) problem has reemerged as an important
theoretical benchmark for reinforcement learning-based control of complex
dynamical systems with continuous state and action spaces. In contrast with
nearly all recent work in this area, we consider multiplicative noise models,
which are increasingly relevant because they explicitly incorporate inherent
uncertainty and variation in the system dynamics and thereby improve robustness
properties of the controller. Robustness is a critical and poorly understood
issue in reinforcement learning; existing methods which do not account for
uncertainty can converge to fragile policies or fail to converge at all.
Additionally, intentional injection of multiplicative noise into learning
algorithms can enhance robustness of policies, as observed in ad hoc work on
domain randomization. Although policy gradient algorithms require optimization
of a non-convex cost function, we show that the multiplicative noise LQR cost
has a special property called gradient domination, which is exploited to prove
global convergence of policy gradient algorithms to the globally optimum
control policy with polynomial dependence on problem parameters. Results are
provided both in the model-known and model-unknown settings where samples of
system trajectories are used to estimate policy gradients.
