Model-based Exploration of the Frontier of Behaviours for Deep Learning
  System Testing
  With the increasing adoption of Deep Learning (DL) for critical tasks, such
as autonomous driving, the evaluation of the quality of systems that rely on DL
has become crucial. Once trained, DL systems produce an output for any
arbitrary numeric vector provided as input, regardless of whether it is within
or outside the validity domain of the system under test. Hence, the quality of
such systems is determined by the intersection between their validity domain
and the regions where their outputs exhibit a misbehaviour. In this paper, we
introduce the notion of frontier of behaviours, i.e., the inputs at which the
DL system starts to misbehave. If the frontier of misbehaviours is outside the
validity domain of the system, the quality check is passed. Otherwise, the
inputs at the intersection represent quality deficiencies of the system. We
developed DeepJanus, a search-based tool that generates frontier inputs for DL
systems. The experimental results obtained for the lane keeping component of a
self-driving car show that the frontier of a well trained system contains
almost exclusively unrealistic roads that violate the best practices of civil
engineering, while the frontier of a poorly trained one includes many valid
inputs that point to serious deficiencies of the system.
