Neural Lyapunov Redesign
  Learning controllers merely based on a performance metric has been proven
effective in many physical and non-physical tasks in both control theory and
reinforcement learning. However, in practice, the controller must guarantee
some notion of safety to ensure that it does not harm either the agent or the
environment. Stability is a crucial notion of safety, whose violation can
certainly cause unsafe behaviors. Lyapunov functions are effective tools to
assess stability in nonlinear dynamical systems. In this paper, we combine an
improving Lyapunov function with automatic controller synthesis in an iterative
fashion to obtain control policies with large safe regions. We propose a
two-player collaborative algorithm that alternates between estimating a
Lyapunov function and deriving a controller that gradually enlarges the
stability region of the closed-loop system. We provide theoretical results on
the class of systems that can be treated with the proposed algorithm and
empirically evaluate the effectiveness of our method using an exemplary
dynamical system.
