Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle
  Validation
  Determining possible failure scenarios is a critical step in the evaluation
of autonomous vehicle systems. Real-world vehicle testing is commonly employed
for autonomous vehicle validation, but the costs and time requirements are
high. Consequently, simulation-driven methods such as Adaptive Stress Testing
(AST) have been proposed to aid in validation. AST formulates the problem of
finding the most likely failure scenarios as a Markov decision process, which
can be solved using reinforcement learning. In practice, AST tends to find
scenarios where failure is unavoidable and tends to repeatedly discover the
same types of failures of a system. This work addresses these issues by
encoding domain relevant information into the search procedure. With this
modification, the AST method discovers a larger and more expressive subset of
the failure space when compared to the original AST formulation. We show that
our approach is able to identify useful failure scenarios of an autonomous
vehicle policy.
