Intra-Processing Methods for Debiasing Neural Networks
  As deep learning models become tasked with more and more decisions that
impact human lives, such as criminal recidivism, loan repayment, and face
recognition for law enforcement, bias is becoming a growing concern. Debiasing
algorithms are typically split into three paradigms: pre-processing,
in-processing, and post-processing. However, in computer vision or natural
language applications, it is common to start with a large generic model and
then fine-tune to a specific use-case. Pre- or in-processing methods would
require retraining the entire model from scratch, while post-processing methods
only have black-box access to the model, so they do not leverage the weights of
the trained model. Creating debiasing algorithms specifically for this
fine-tuning use-case has largely been neglected.
  In this work, we initiate the study of a new paradigm in debiasing research,
intra-processing, which sits between in-processing and post-processing methods.
Intra-processing methods are designed specifically to debias large models which
have been trained on a generic dataset and fine-tuned on a more specific task.
We show how to repurpose existing in-processing methods for this use-case, and
we also propose three baseline algorithms: random perturbation, layerwise
optimization, and adversarial fine-tuning. All of our techniques can be used
for all popular group fairness measures such as equalized odds or statistical
parity difference. We evaluate these methods across three popular datasets from
the AIF360 toolkit, as well as on the CelebA faces dataset. Our code is
available at https://github.com/abacusai/intraprocessing_debiasing.
