Reliability Assessment and Safety Arguments for Machine Learning
  Components in Assuring Learning-Enabled Autonomous Systems
  The increasing use of Machine Learning (ML) components embedded in autonomous
systems -- so-called Learning-Enabled Systems (LES) -- has resulted in the
pressing need to assure their functional safety. As for traditional functional
safety, the emerging consensus within both, industry and academia, is to use
assurance cases for this purpose. Typically assurance cases support claims of
reliability in support of safety, and can be viewed as a structured way of
organising arguments and evidence generated from safety analysis and
reliability modelling activities. While such assurance activities are
traditionally guided by consensus-based standards developed from vast
engineering experience, LES pose new challenges in safety-critical application
due to the characteristics and design of ML models. In this article, we first
present an overall assurance framework for LES with an emphasis on quantitative
aspects, e.g., breaking down system-level safety targets to component-level
requirements and supporting claims stated in reliability metrics. We then
introduce a novel model-agnostic Reliability Assessment Model (RAM) for ML
classifiers that utilises the operational profile and robustness verification
evidence. We discuss the model assumptions and the inherent challenges of
assessing ML reliability uncovered by our RAM and propose practical solutions.
Probabilistic safety arguments at the lower ML component-level are also
developed based on the RAM. Finally, to evaluate and demonstrate our methods,
we not only conduct experiments on synthetic/benchmark datasets but also
demonstrate the scope of our methods with a comprehensive case study on
Autonomous Underwater Vehicles in simulation.
