Robust Learning-based Predictive Control for Discrete-time Nonlinear
  Systems with Unknown Dynamics and State Constraints
  Robust model predictive control (MPC) is a well-known control technique for
model-based control with constraints and uncertainties. In classic robust
tube-based MPC approaches, an open-loop control sequence is computed via
periodically solving an online nominal MPC problem, which requires prior model
information and frequent access to onboard computational resources. In this
paper, we propose an efficient robust MPC solution based on receding horizon
reinforcement learning, called r-LPC, for unknown nonlinear systems with state
constraints and disturbances. The proposed r-LPC utilizes a Koopman
operator-based prediction model obtained off-line from pre-collected
input-output datasets. Unlike classic tube-based MPC, in each prediction time
interval of r-LPC, we use an actor-critic structure to learn a near-optimal
feedback control policy rather than a control sequence. The resulting
closed-loop control policy can be learned off-line and deployed online or
learned online in an asynchronous way. In the latter case, online learning can
be activated whenever necessary; for instance, the safety constraint is
violated with the deployed policy. The closed-loop recursive feasibility,
robustness, and asymptotic stability are proven under function approximation
errors of the actor-critic networks. Simulation and experimental results on two
nonlinear systems with unknown dynamics and disturbances have demonstrated that
our approach has better or comparable performance when compared with tube-based
MPC and LQR, and outperforms a recently developed actor-critic learning
approach.
