Towards Safe Machine Learning for CPS: Infer Uncertainty from Training
  Data
  Machine learning (ML) techniques are increasingly applied to decision-making
and control problems in Cyber-Physical Systems among which many are
safety-critical, e.g., chemical plants, robotics, autonomous vehicles. Despite
the significant benefits brought by ML techniques, they also raise additional
safety issues because 1) most expressive and powerful ML models are not
transparent and behave as a black box and 2) the training data which plays a
crucial role in ML safety is usually incomplete. An important technique to
achieve safety for ML models is "Safe Fail", i.e., a model selects a reject
option and applies the backup solution, a traditional controller or a human
operator for example, when it has low confidence in a prediction.
  Data-driven models produced by ML algorithms learn from training data, and
hence they are only as good as the examples they have learnt. As pointed in
[17], ML models work well in the "training space" (i.e., feature space with
sufficient training data), but they could not extrapolate beyond the training
space. As observed in many previous studies, a feature space that lacks
training data generally has a much higher error rate than the one that contains
sufficient training samples [31]. Therefore, it is essential to identify the
training space and avoid extrapolating beyond the training space. In this
paper, we propose an efficient Feature Space Partitioning Tree (FSPT) to
address this problem. Using experiments, we also show that, a strong
relationship exists between model performance and FSPT score.
