Customizable Reference Runtime Monitoring of Neural Networks using
  Resolution Boxes
  Classification neural networks fail to detect inputs that do not fall inside
the classes they have been trained for. Runtime monitoring techniques on the
neuron activation pattern can be used to detect such inputs. We present an
approach for monitoring classification systems via data abstraction. Data
abstraction relies on the notion of box with a resolution. Box-based
abstraction consists in representing a set of values by its minimal and maximal
values in each dimension. We augment boxes with a notion of resolution and
define their clustering coverage, which is intuitively a quantitative metric
that indicates the abstraction quality. This allows studying the effect of
different clustering parameters on the constructed boxes and estimating an
interval of sub-optimal parameters. Moreover, we automatically construct
monitors that leverage both the correct and incorrect behaviors of a system.
This allows checking the size of the monitor abstractions and analyzing the
separability of the network. Monitors are obtained by combining the
sub-monitors of each class of the system placed at some selected layers. Our
experiments demonstrate the effectiveness of our clustering coverage estimation
and show how to assess the effectiveness and precision of monitors according to
the selected clustering parameter and monitored layers.
