Social Norm Bias: Residual Harms of Fairness-Aware Algorithms
  Many modern learning algorithms mitigate bias by enforcing fairness across
coarsely-defined groups related to a sensitive attribute like gender or race.
However, the same algorithms seldom account for the within-group biases that
arise due to the heterogeneity of group members. In this work, we characterize
Social Norm Bias (SNoB), a subtle but consequential type of discrimination that
may be exhibited by automated decision-making systems, even when these systems
achieve group fairness objectives. We study this issue through the lens of
gender bias in occupation classification from biographies. We quantify SNoB by
measuring how an algorithm's predictions are associated with conformity to
gender norms, which is measured using a machine learning approach. This
framework reveals that for classification tasks related to male-dominated
occupations, fairness-aware classifiers favor biographies written in ways that
align with masculine gender norms. We compare SNoB across fairness intervention
techniques and show that post-processing interventions do not mitigate this
type of bias at all.
