Civil Rephrases Of Toxic Texts With Self-Supervised Transformers
  Platforms that support online commentary, from social networks to news sites,
are increasingly leveraging machine learning to assist their moderation
efforts. But this process does not typically provide feedback to the author
that would help them contribute according to the community guidelines. This is
prohibitively time-consuming for human moderators to do, and computational
approaches are still nascent. This work focuses on models that can help suggest
rephrasings of toxic comments in a more civil manner. Inspired by recent
progress in unpaired sequence-to-sequence tasks, a self-supervised learning
model is introduced, called CAE-T5. CAE-T5 employs a pre-trained text-to-text
transformer, which is fine tuned with a denoising and cyclic auto-encoder loss.
Experimenting with the largest toxicity detection dataset to date (Civil
Comments) our model generates sentences that are more fluent and better at
preserving the initial content compared to earlier text style transfer systems
which we compare with using several scoring systems and human evaluation.
