Neural Networks for Text Correction and Completion in Keyboard Decoding
  Despite the ubiquity of mobile and wearable text messaging applications, the
problem of keyboard text decoding is not tackled sufficiently in the light of
the enormous success of the deep learning Recurrent Neural Network (RNN) and
Convolutional Neural Networks (CNN) for natural language understanding. In
particular, considering that the keyboard decoders should operate on devices
with memory and processor resource constraints, makes it challenging to deploy
industrial scale deep neural network (DNN) models. This paper proposes a
sequence-to-sequence neural attention network system for automatic text
correction and completion. Given an erroneous sequence, our model encodes
character level hidden representations and then decodes the revised sequence
thus enabling auto-correction and completion. We achieve this by a combination
of character level CNN and gated recurrent unit (GRU) encoder along with and a
word level gated recurrent unit (GRU) attention decoder. Unlike traditional
language models that learn from billions of words, our corpus size is only 12
million words; an order of magnitude smaller. The memory footprint of our
learnt model for inference and prediction is also an order of magnitude smaller
than the conventional language model based text decoders. We report baseline
performance for neural keyboard decoders in such limited domain. Our models
achieve a word level accuracy of $90\%$ and a character error rate CER of
$2.4\%$ over the Twitter typo dataset. We present a novel dataset of noisy to
corrected mappings by inducing the noise distribution from the Twitter data
over the OpenSubtitles 2009 dataset; on which our model predicts with a word
level accuracy of $98\%$ and sequence accuracy of $68.9\%$. In our user study,
our model achieved an average CER of $2.6\%$ with the state-of-the-art
non-neural touch-screen keyboard decoder at CER of $1.6\%$.
