A Character-Level Approach to the Text Normalization Problem Based on a
  New Causal Encoder
  Text normalization is a ubiquitous process that appears as the first step of
many Natural Language Processing problems. However, previous Deep Learning
approaches have suffered from so-called silly errors, which are undetectable on
unsupervised frameworks, making those models unsuitable for deployment. In this
work, we make use of an attention-based encoder-decoder architecture that
overcomes these undetectable errors by using a fine-grained character-level
approach rather than a word-level one. Furthermore, our new general-purpose
encoder based on causal convolutions, called Causal Feature Extractor (CFE), is
introduced and compared to other common encoders. The experimental results show
the feasibility of this encoder, which leverages the attention mechanisms the
most and obtains better results in terms of accuracy, number of parameters and
convergence time. While our method results in a slightly worse initial accuracy
(92.74%), errors can be automatically detected and, thus, more readily solved,
obtaining a more robust model for deployment. Furthermore, there is still
plenty of room for future improvements that will push even further these
advantages.
