On the Effectiveness of Iterative Learning Control
  Iterative learning control (ILC) is a powerful technique for high performance
tracking in the presence of modeling errors for optimal control applications.
There is extensive prior work showing its empirical effectiveness in
applications such as chemical reactors, industrial robots and quadcopters.
However, there is little prior theoretical work that explains the effectiveness
of ILC even in the presence of large modeling errors, where optimal control
methods using the misspecified model (MM) often perform poorly. Our work
presents such a theoretical study of the performance of both ILC and MM on
Linear Quadratic Regulator (LQR) problems with unknown transition dynamics. We
show that the suboptimality gap, as measured with respect to the optimal LQR
controller, for ILC is lower than that for MM by higher order terms that become
significant in the regime of high modeling errors. A key part of our analysis
is the perturbation bounds for the discrete Ricatti equation in the finite
horizon setting, where the solution is not a fixed point and requires tracking
the error using recursive bounds. We back our theoretical findings with
empirical experiments on a toy linear dynamical system with an approximate
model, a nonlinear inverted pendulum system with misspecified mass, and a
nonlinear planar quadrotor system in the presence of wind. Experiments show
that ILC outperforms MM significantly, in terms of the cost of computed
trajectories, when modeling errors are high.
