Mitigating harm in language models with conditional-likelihood
  filtration
  Language models trained on large-scale unfiltered datasets curated from the
open web acquire systemic biases, prejudices, and harmful views from their
training data. We present a methodology for programmatically identifying and
removing harmful text from web-scale datasets. A pretrained language model is
used to calculate the log-likelihood of researcher-written trigger phrases
conditioned on a specific document, which is used to identify and filter
documents from the dataset. We demonstrate that models trained on this filtered
dataset exhibit lower propensity to generate harmful text, with a marginal
decrease in performance on standard language modeling benchmarks compared to
unfiltered baselines. We provide a partial explanation for this performance gap
by surfacing examples of hate speech and other undesirable content from
standard language modeling benchmarks. Finally, we discuss the generalization
of this method and how trigger phrases which reflect specific values can be
used by researchers to build language models which are more closely aligned
with their values.
