Assurance Monitoring of Learning Enabled Cyber-Physical Systems Using
  Inductive Conformal Prediction based on Distance Learning
  Machine learning components such as deep neural networks are used extensively
in Cyber-Physical Systems (CPS). However, such components may introduce new
types of hazards that can have disastrous consequences and need to be addressed
for engineering trustworthy systems. Although deep neural networks offer
advanced capabilities, they must be complemented by engineering methods and
practices that allow effective integration in CPS. In this paper, we proposed
an approach for assurance monitoring of learning-enabled CPS based on the
conformal prediction framework. In order to allow real-time assurance
monitoring, the approach employs distance learning to transform
high-dimensional inputs into lower size embedding representations. By
leveraging conformal prediction, the approach provides well-calibrated
confidence and ensures a bounded small error rate while limiting the number of
inputs for which an accurate prediction cannot be made. We demonstrate the
approach using three data sets of mobile robot following a wall, speaker
recognition, and traffic sign recognition. The experimental results demonstrate
that the error rates are well-calibrated while the number of alarms is very
small. Further, the method is computationally efficient and allows real-time
assurance monitoring of CPS.
