Assuring the Machine Learning Lifecycle: Desiderata, Methods, and
  Challenges
  Machine learning has evolved into an enabling technology for a wide range of
highly successful applications. The potential for this success to continue and
accelerate has placed machine learning (ML) at the top of research, economic
and political agendas. Such unprecedented interest is fuelled by a vision of ML
applicability extending to healthcare, transportation, defence and other
domains of great societal importance. Achieving this vision requires the use of
ML in safety-critical applications that demand levels of assurance beyond those
needed for current ML applications. Our paper provides a comprehensive survey
of the state-of-the-art in the assurance of ML, i.e. in the generation of
evidence that ML is sufficiently safe for its intended use. The survey covers
the methods capable of providing such evidence at different stages of the
machine learning lifecycle, i.e. of the complex, iterative process that starts
with the collection of the data used to train an ML component for a system, and
ends with the deployment of that component within the system. The paper begins
with a systematic presentation of the ML lifecycle and its stages. We then
define assurance desiderata for each stage, review existing methods that
contribute to achieving these desiderata, and identify open challenges that
require further research.
