Detecting model drift using polynomial relations
  Machine learning models serve critical functions, such as classifying loan
applicants as good or bad risks. Each model is trained under the assumption
that the data used in training and in the field come from the same underlying
unknown distribution. Often, this assumption is broken in practice. It is
desirable to identify when this occurs, to minimize the impact on model
performance.
  We suggest a new approach to detecting change in the data distribution by
identifying polynomial relations between the data features. We measure the
strength of each identified relation using its R-square value. A strong
polynomial relation captures a significant trait of the data which should
remain stable if the data distribution does not change. We thus use a set of
learned strong polynomial relations to identify drift. For a set of polynomial
relations that are stronger than a given threshold, we calculate the amount of
drift observed for that relation. The amount of drift is measured by
calculating the Bayes Factor for the polynomial relation likelihood of the
baseline data versus field data. We empirically validate the approach by
simulating a range of changes, and identify drift using the Bayes Factor of the
polynomial relation likelihood change.
