Algorithmic Recourse in the Wild: Understanding the Impact of Data and
  Model Shifts
  As predictive models are increasingly being deployed to make a variety of
consequential decisions, there is a growing emphasis on designing algorithms
that can provide recourse to affected individuals. Existing recourse algorithms
function under the assumption that the underlying predictive model does not
change. However, models are regularly updated in practice for several reasons
including data distribution shifts. In this work, we make the first attempt at
understanding how model updates resulting from data distribution shifts impact
the algorithmic recourses generated by state-of-the-art algorithms. We carry
out a rigorous theoretical and empirical analysis to address the above
question. Our theoretical results establish a lower bound on the probability of
recourse invalidation due to model shifts, and show the existence of a tradeoff
between this invalidation probability and typical notions of "cost" minimized
by modern recourse generation algorithms. We experiment with multiple synthetic
and real world datasets, capturing different kinds of distribution shifts
including temporal shifts, geospatial shifts, and shifts due to data
correction. These experiments demonstrate that model updation due to all the
aforementioned distribution shifts can potentially invalidate recourses
generated by state-of-the-art algorithms. Our findings thus not only expose
previously unknown flaws in the current recourse generation paradigm, but also
pave the way for fundamentally rethinking the design and development of
recourse generation algorithms.
