Transfer of Machine Learning Fairness across Domains
  If our models are used in new or unexpected cases, do we know if they will
make fair predictions? Previously, researchers developed ways to debias a model
for a single problem domain. However, this is often not how models are trained
and used in practice. For example, labels and demographics (sensitive
attributes) are often hard to observe, resulting in auxiliary or synthetic data
to be used for training, and proxies of the sensitive attribute to be used for
evaluation of fairness. A model trained for one setting may be picked up and
used in many others, particularly as is common with pre-training and cloud
APIs. Despite the pervasiveness of these complexities, remarkably little work
in the fairness literature has theoretically examined these issues. We frame
all of these settings as domain adaptation problems: how can we use what we
have learned in a source domain to debias in a new target domain, without
directly debiasing on the target domain as if it is a completely new problem?
We offer new theoretical guarantees of improving fairness across domains, and
offer a modeling approach to transfer to data-sparse target domains. We give
empirical results validating the theory and showing that these modeling
approaches can improve fairness metrics with less data.
