Unintended Selection: Persistent Qualification Rate Disparities and
  Interventions
  Realistically -- and equitably -- modeling the dynamics of group-level
disparities in machine learning remains an open problem. In particular, we
desire models that do not suppose inherent differences between artificial
groups of people -- but rather endogenize disparities by appeal to unequal
initial conditions of insular subpopulations. In this paper, agents each have a
real-valued feature $X$ (e.g., credit score) informed by a "true" binary label
$Y$ representing qualification (e.g., for a loan). Each agent alternately (1)
receives a binary classification label $\hat{Y}$ (e.g., loan approval) from a
Bayes-optimal machine learning classifier observing $X$ and (2) may update
their qualification $Y$ by imitating successful strategies (e.g., seek a raise)
within an isolated group $G$ of agents to which they belong. We consider the
disparity of qualification rates $\Pr(Y=1)$ between different groups and how
this disparity changes subject to a sequence of Bayes-optimal classifiers
repeatedly retrained on the global population. We model the evolving
qualification rates of each subpopulation (group) using the replicator
equation, which derives from a class of imitation processes. We show that
differences in qualification rates between subpopulations can persist
indefinitely for a set of non-trivial equilibrium states due to uniformed
classifier deployments, even when groups are identical in all aspects except
initial qualification densities. We next simulate the effects of commonly
proposed fairness interventions on this dynamical system along with a new
feedback control mechanism capable of permanently eliminating group-level
qualification rate disparities. We conclude by discussing the limitations of
our model and findings and by outlining potential future work.
