Koopman Spectrum Nonlinear Regulator and Provably Efficient Online
  Learning
  Most modern reinforcement learning algorithms optimize a cumulative
single-step cost along a trajectory. The optimized motions are often
'unnatural', representing, for example, behaviors with sudden accelerations
that waste energy and lack predictability. In this work, we present a novel
paradigm of controlling nonlinear systems via the minimization of the Koopman
spectrum cost: a cost over the Koopman operator of the controlled dynamics.
This induces a broader class of dynamical behaviors that evolve over stable
manifolds such as nonlinear oscillators, closed loops, and smooth movements. We
demonstrate that some dynamics realizations that are not possible with a
cumulative cost are feasible in this paradigm. Moreover, we present a provably
efficient online learning algorithm for our problem that enjoys a sub-linear
regret bound under some structural assumptions.
