Learning to Reach, Swim, Walk and Fly in One Trial: Data-Driven Control
  with Scarce Data and Side Information
  We develop a learning-based control algorithm for unknown dynamical systems
under very severe data limitations. Specifically, the algorithm has access to
streaming and noisy data only from a single and ongoing trial. It accomplishes
such performance by effectively leveraging various forms of side information on
the dynamics to reduce the sample complexity. Such side information typically
comes from elementary laws of physics and qualitative properties of the system.
More precisely, the algorithm approximately solves an optimal control problem
encoding the system's desired behavior. To this end, it constructs and
iteratively refines a data-driven differential inclusion that contains the
unknown vector field of the dynamics. The differential inclusion, used in an
interval Taylor-based method, enables to over-approximate the set of states the
system may reach. Theoretically, we establish a bound on the suboptimality of
the approximate solution with respect to the optimal control with known
dynamics. We show that the longer the trial or the more side information is
available, the tighter the bound. Empirically, experiments in a high-fidelity
F-16 aircraft simulator and MuJoCo's environments illustrate that, despite the
scarcity of data, the algorithm can provide performance comparable to
reinforcement learning algorithms trained over millions of environment
interactions. Besides, we show that the algorithm outperforms existing
techniques combining system identification and model predictive control.
