Machine Learning Testing in an ADAS Case Study Using
  Simulation-Integrated Bio-Inspired Search-Based Testing
  This paper presents an extended version of Deeper, a search-based
simulation-integrated test solution that generates failure-revealing test
scenarios for testing a deep neural network-based lane-keeping system. In the
newly proposed version, we utilize a new set of bio-inspired search algorithms,
genetic algorithm (GA), $({\mu}+{\lambda})$ and $({\mu},{\lambda})$ evolution
strategies (ES), and particle swarm optimization (PSO), that leverage a quality
population seed and domain-specific cross-over and mutation operations tailored
for the presentation model used for modeling the test scenarios. In order to
demonstrate the capabilities of the new test generators within Deeper, we carry
out an empirical evaluation and comparison with regard to the results of five
participating tools in the cyber-physical systems testing competition at SBST
2021. Our evaluation shows the newly proposed test generators in Deeper not
only represent a considerable improvement on the previous version but also
prove to be effective and efficient in provoking a considerable number of
diverse failure-revealing test scenarios for testing an ML-driven lane-keeping
system. They can trigger several failures while promoting test scenario
diversity, under a limited test time budget, high target failure severity, and
strict speed limit constraints.
