Neural Network Repair with Reachability Analysis
  Safety is a critical concern for the next generation of autonomy that is
likely to rely heavily on deep neural networks for perception and control.
Formally verifying the safety and robustness of well-trained DNNs and
learning-enabled systems under attacks, model uncertainties, and sensing errors
is essential for safe autonomy. This research proposes a framework to repair
unsafe DNNs in safety-critical systems with reachability analysis. The repair
process is inspired by adversarial training which has demonstrated high
effectiveness in improving the safety and robustness of DNNs. Different from
traditional adversarial training approaches where adversarial examples are
utilized from random attacks and may not be representative of all unsafe
behaviors, our repair process uses reachability analysis to compute the exact
unsafe regions and identify sufficiently representative examples to enhance the
efficacy and efficiency of the adversarial training.
  The performance of our framework is evaluated on two types of benchmarks
without safe models as references. One is a DNN controller for aircraft
collision avoidance with access to training data. The other is a rocket lander
where our framework can be seamlessly integrated with the well-known deep
deterministic policy gradient (DDPG) reinforcement learning algorithm. The
experimental results show that our framework can successfully repair all
instances on multiple safety specifications with negligible performance
degradation. In addition, to increase the computational and memory efficiency
of the reachability analysis algorithm, we propose a depth-first-search
algorithm that combines an existing exact analysis method with an
over-approximation approach based on a new set representation. Experimental
results show that our method achieves a five-fold improvement in runtime and a
two-fold improvement in memory usage compared to exact analysis.
