Model-based Constrained Reinforcement Learning using Generalized Control
  Barrier Function
  Model information can be used to predict future trajectories, so it has huge
potential to avoid dangerous region when implementing reinforcement learning
(RL) on real-world tasks, like autonomous driving. However, existing studies
mostly use model-free constrained RL, which causes inevitable constraint
violations. This paper proposes a model-based feasibility enhancement technique
of constrained RL, which enhances the feasibility of policy using generalized
control barrier function (GCBF) defined on the distance to constraint boundary.
By using the model information, the policy can be optimized safely without
violating actual safety constraints, and the sample efficiency is increased.
The major difficulty of infeasibility in solving the constrained policy
gradient is handled by an adaptive coefficient mechanism. We evaluate the
proposed method in both simulations and real vehicle experiments in a complex
autonomous driving collision avoidance task. The proposed method achieves up to
four times fewer constraint violations and converges 3.36 times faster than
baseline constrained RL approaches.
