Reinforcement Learning for Safety-Critical Control under Model
  Uncertainty, using Control Lyapunov Functions and Control Barrier Functions
  In this paper, the issue of model uncertainty in safety-critical control is
addressed with a data-driven approach. For this purpose, we utilize the
structure of an input-ouput linearization controller based on a nominal model
along with a Control Barrier Function and Control Lyapunov Function based
Quadratic Program (CBF-CLF-QP). Specifically, we propose a novel reinforcement
learning framework which learns the model uncertainty present in the CBF and
CLF constraints, as well as other control-affine dynamic constraints in the
quadratic program. The trained policy is combined with the nominal model-based
CBF-CLF-QP, resulting in the Reinforcement Learning-based CBF-CLF-QP
(RL-CBF-CLF-QP), which addresses the problem of model uncertainty in the safety
constraints. The performance of the proposed method is validated by testing it
on an underactuated nonlinear bipedal robot walking on randomly spaced stepping
stones with one step preview, obtaining stable and safe walking under model
uncertainty.
