ShieldNN: A Provably Safe NN Filter for Unsafe NN Controllers
  In this paper, we consider the problem of creating a safe-by-design Rectified
Linear Unit (ReLU) Neural Network (NN), which, when composed with an arbitrary
control NN, makes the composition provably safe. In particular, we propose an
algorithm to synthesize such NN filters that safely correct control inputs
generated for the continuous-time Kinematic Bicycle Model (KBM). ShieldNN
contains two main novel contributions: first, it is based on a novel Barrier
Function (BF) for the KBM model; and second, it is itself a provably sound
algorithm that leverages this BF to a design a safety filter NN with safety
guarantees. Moreover, since the KBM is known to well approximate the dynamics
of four-wheeled vehicles, we show the efficacy of ShieldNN filters in CARLA
simulations of four-wheeled vehicles. In particular, we examined the effect of
ShieldNN filters on Deep Reinforcement Learning trained controllers in the
presence of individual pedestrian obstacles. The safety properties of ShieldNN
were borne out in our experiments: the ShieldNN filter reduced the number of
obstacle collisions by 99.4%-100%. Furthermore, we also studied the effect of
incorporating ShieldNN during training: for a constant number of episodes, 28%
less reward was observed when ShieldNN wasn't used during training. This
suggests that ShieldNN has the further property of improving sample efficiency
during RL training.
