Demographics Should Not Be the Reason of Toxicity: Mitigating
  Discrimination in Text Classifications with Instance Weighting
  With the recent proliferation of the use of text classifications, researchers
have found that there are certain unintended biases in text classification
datasets. For example, texts containing some demographic identity-terms (e.g.,
"gay", "black") are more likely to be abusive in existing abusive language
detection datasets. As a result, models trained with these datasets may
consider sentences like "She makes me happy to be gay" as abusive simply
because of the word "gay." In this paper, we formalize the unintended biases in
text classification datasets as a kind of selection bias from the
non-discrimination distribution to the discrimination distribution. Based on
this formalization, we further propose a model-agnostic debiasing training
framework by recovering the non-discrimination distribution using instance
weighting, which does not require any extra resources or annotations apart from
a pre-defined set of demographic identity-terms. Experiments demonstrate that
our method can effectively alleviate the impacts of the unintended biases
without significantly hurting models' generalization ability.
