RECAST: Enabling User Recourse and Interpretability of Toxicity
  Detection Models with Interactive Visualization
  With the widespread use of toxic language online, platforms are increasingly
using automated systems that leverage advances in natural language processing
to automatically flag and remove toxic comments. However, most automated
systems -- when detecting and moderating toxic language -- do not provide
feedback to their users, let alone provide an avenue of recourse for these
users to make actionable changes. We present our work, RECAST, an interactive,
open-sourced web tool for visualizing these models' toxic predictions, while
providing alternative suggestions for flagged toxic language. Our work also
provides users with a new path of recourse when using these automated
moderation tools. RECAST highlights text responsible for classifying toxicity,
and allows users to interactively substitute potentially toxic phrases with
neutral alternatives. We examined the effect of RECAST via two large-scale user
evaluations, and found that RECAST was highly effective at helping users reduce
toxicity as detected through the model. Users also gained a stronger
understanding of the underlying toxicity criterion used by black-box models,
enabling transparency and recourse. In addition, we found that when users focus
on optimizing language for these models instead of their own judgement (which
is the implied incentive and goal of deploying automated models), these models
cease to be effective classifiers of toxicity compared to human annotations.
This opens a discussion for how toxicity detection models work and should work,
and their effect on the future of online discourse.
