Mitigating Gender Bias Amplification in Distribution by Posterior
  Regularization
  Advanced machine learning techniques have boosted the performance of natural
language processing. Nevertheless, recent studies, e.g., Zhao et al. (2017)
show that these techniques inadvertently capture the societal bias hidden in
the corpus and further amplify it. However, their analysis is conducted only on
models' top predictions. In this paper, we investigate the gender bias
amplification issue from the distribution perspective and demonstrate that the
bias is amplified in the view of predicted probability distribution over
labels. We further propose a bias mitigation approach based on posterior
regularization. With little performance loss, our method can almost remove the
bias amplification in the distribution. Our study sheds the light on
understanding the bias amplification.
