A Safety Framework for Critical Systems Utilising Deep Neural Networks
  Increasingly sophisticated mathematical modelling processes from Machine
Learning are being used to analyse complex data. However, the performance and
explainability of these models within practical critical systems requires a
rigorous and continuous verification of their safe utilisation. Working towards
addressing this challenge, this paper presents a principled novel safety
argument framework for critical systems that utilise deep neural networks. The
approach allows various forms of predictions, e.g., future reliability of
passing some demands, or confidence on a required reliability level. It is
supported by a Bayesian analysis using operational data and the recent
verification and validation techniques for deep learning. The prediction is
conservative -- it starts with partial prior knowledge obtained from lifecycle
activities and then determines the worst-case prediction. Open challenges are
also identified.
