Tracking the risk of a deployed model and detecting harmful distribution
  shifts
  When deployed in the real world, machine learning models inevitably encounter
changes in the data distribution, and certain -- but not all -- distribution
shifts could result in significant performance degradation. In practice, it may
make sense to ignore benign shifts, under which the performance of a deployed
model does not degrade substantially, making interventions by a human expert
(or model retraining) unnecessary. While several works have developed tests for
distribution shifts, these typically either use non-sequential methods, or
detect arbitrary shifts (benign or harmful), or both. We argue that a sensible
method for firing off a warning has to both (a) detect harmful shifts while
ignoring benign ones, and (b) allow continuous monitoring of model performance
without increasing the false alarm rate. In this work, we design simple
sequential tools for testing if the difference between source (training) and
target (test) distributions leads to a significant increase in a risk function
of interest, like accuracy or calibration. Recent advances in constructing
time-uniform confidence sequences allow efficient aggregation of statistical
evidence accumulated during the tracking process. The designed framework is
applicable in settings where (some) true labels are revealed after the
prediction is performed, or when batches of labels become available in a
delayed fashion. We demonstrate the efficacy of the proposed framework through
an extensive empirical study on a collection of simulated and real datasets.
