Into the Unknown: Active Monitoring of Neural Networks
  Neural-network classifiers achieve high accuracy when predicting the class of
an input that they were trained to identify. Maintaining this accuracy in
dynamic environments, where inputs frequently fall outside the fixed set of
initially known classes, remains a challenge. The typical approach is to detect
inputs from novel classes and retrain the classifier on an augmented dataset.
However, not only the classifier but also the detection mechanism needs to
adapt in order to distinguish between newly learned and yet unknown input
classes. To address this challenge, we introduce an algorithmic framework for
active monitoring of a neural network. A monitor wrapped in our framework
operates in parallel with the neural network and interacts with a human user
via a series of interpretable labeling queries for incremental adaptation. In
addition, we propose an adaptive quantitative monitor to improve precision. An
experimental evaluation on a diverse set of benchmarks with varying numbers of
classes confirms the benefits of our active monitoring framework in dynamic
scenarios.
