Model Mis-specification and Algorithmic Bias
  Machine learning algorithms are increasingly used to inform critical
decisions. There is a growing concern about bias, that algorithms may produce
uneven outcomes for individuals in different demographic groups. In this work,
we measure bias as the difference between mean prediction errors across groups.
We show that even with unbiased input data, when a model is mis-specified: (1)
population-level mean prediction error can still be negligible, but group-level
mean prediction errors can be large; (2) such errors are not equal across
groups; and (3) the difference between errors, i.e., bias, can take the
worst-case realization. That is, when there are two groups of the same size,
mean prediction errors for these two groups have the same magnitude but
opposite signs. In closed form, we show such errors and bias are functions of
the first and second moments of the joint distribution of features (for linear
and probit regressions). We also conduct numerical experiments to show similar
results in more general settings. Our work provides a first step for decoupling
the impact of different causes of bias.
