Feedback Detection for Live Predictors
  A predictor that is deployed in a live production system may perturb the
features it uses to make predictions. Such a feedback loop can occur, for
example, when a model that predicts a certain type of behavior ends up causing
the behavior it predicts, thus creating a self-fulfilling prophecy. In this
paper we analyze predictor feedback detection as a causal inference problem,
and introduce a local randomization scheme that can be used to detect
non-linear feedback in real-world problems. We conduct a pilot study for our
proposed methodology using a predictive system currently deployed as a part of
a search engine.
