Finding Failures in High-Fidelity Simulation using Adaptive Stress
  Testing and the Backward Algorithm
  Validating the safety of autonomous systems generally requires the use of
high-fidelity simulators that adequately capture the variability of real-world
scenarios. However, it is generally not feasible to exhaustively search the
space of simulation scenarios for failures. Adaptive stress testing (AST) is a
method that uses reinforcement learning to find the most likely failure of a
system. AST with a deep reinforcement learning solver has been shown to be
effective in finding failures across a range of different systems. This
approach generally involves running many simulations, which can be very
expensive when using a high-fidelity simulator. To improve efficiency, we
present a method that first finds failures in a low-fidelity simulator. It then
uses the backward algorithm, which trains a deep neural network policy using a
single expert demonstration, to adapt the low-fidelity failures to
high-fidelity. We have created a series of autonomous vehicle validation case
studies that represent some of the ways low-fidelity and high-fidelity
simulators can differ, such as time discretization. We demonstrate in a variety
of case studies that this new AST approach is able to find failures with
significantly fewer high-fidelity simulation steps than are needed when just
running AST directly in high-fidelity. As a proof of concept, we also
demonstrate AST on NVIDIA's DriveSim simulator, an industry state-of-the-art
high-fidelity simulator for finding failures in autonomous vehicles.
