Towards Guidelines for Assessing Qualities of Machine Learning Systems
  Nowadays, systems containing components based on machine learning (ML)
methods are becoming more widespread. In order to ensure the intended behavior
of a software system, there are standards that define necessary quality aspects
of the system and its components (such as ISO/IEC 25010). Due to the different
nature of ML, we have to adjust quality aspects or add additional ones (such as
trustworthiness) and be very precise about which aspect is really relevant for
which object of interest (such as completeness of training data), and how to
objectively assess adherence to quality requirements. In this article, we
present the construction of a quality model (i.e., evaluation objects, quality
aspects, and metrics) for an ML system based on an industrial use case. This
quality model enables practitioners to specify and assess quality requirements
for such kinds of ML systems objectively. In the future, we want to learn how
the term quality differs between different types of ML systems and come up with
general guidelines for specifying and assessing qualities of ML systems.
