Spelling Correction with Denoising Transformer
  We present a novel method of performing spelling correction on short input
strings, such as search queries or individual words. At its core lies a
procedure for generating artificial typos which closely follow the error
patterns manifested by humans. This procedure is used to train the production
spelling correction model based on a transformer architecture. This model is
currently served in the HubSpot product search. We show that our approach to
typo generation is superior to the widespread practice of adding noise, which
ignores human patterns. We also demonstrate how our approach may be extended to
resource-scarce settings and train spelling correction models for Arabic,
Greek, Russian, and Setswana languages, without using any labeled data.
