Online Learning for Predictive Control with Provable Regret Guarantees
  We study the problem of online learning in predictive control of an unknown
linear dynamical system with time varying cost functions which are unknown
apriori. Specifically, we study the online learning problem where the control
algorithm does not know the true system model and has only access to a
fixed-length (that does not grow with the control horizon) preview of the
future cost functions. The goal of the online algorithm is to minimize the
dynamic regret, defined as the difference between the cumulative cost incurred
by the algorithm and that of the best sequence of actions in hindsight. We
propose two different online Model Predictive Control (MPC) algorithms to
address this problem, namely Certainty Equivalence MPC (CE-MPC) algorithm and
Optimistic MPC (O-MPC) algorithm. We show that under the standard stability
assumption for the model estimate, the CE-MPC algorithm achieves
$\mathcal{O}(T^{2/3})$ dynamic regret. We then extend this result to the
setting where the stability assumption holds only for the true system model by
proposing the O-MPC algorithm. We show that the O-MPC algorithm also achieves
$\mathcal{O}(T^{2/3})$ dynamic regret, at the cost of some additional
computation. We also present numerical studies to demonstrate the performance
of our algorithm.
