Training Adversarial Agents to Exploit Weaknesses in Deep Control
  Policies
  Deep learning has become an increasingly common technique for various control
problems, such as robotic arm manipulation, robot navigation, and autonomous
vehicles. However, the downside of using deep neural networks to learn control
policies is their opaque nature and the difficulties of validating their
safety. As the networks used to obtain state-of-the-art results become
increasingly deep and complex, the rules they have learned and how they operate
become more challenging to understand. This presents an issue, since in
safety-critical applications the safety of the control policy must be ensured
to a high confidence level. In this paper, we propose an automated black box
testing framework based on adversarial reinforcement learning. The technique
uses an adversarial agent, whose goal is to degrade the performance of the
target model under test. We test the approach on an autonomous vehicle problem,
by training an adversarial reinforcement learning agent, which aims to cause a
deep neural network-driven autonomous vehicle to collide. Two neural networks
trained for autonomous driving are compared, and the results from the testing
are used to compare the robustness of their learned control policies. We show
that the proposed framework is able to find weaknesses in both control policies
that were not evident during online testing and therefore, demonstrate a
significant benefit over manual testing methods.
