An Error-Oriented Approach to Word Embedding Pre-Training
  We propose a novel word embedding pre-training approach that exploits writing
errors in learners' scripts. We compare our method to previous models that tune
the embeddings based on script scores and the discrimination between correct
and corrupt word contexts in addition to the generic commonly-used embeddings
pre-trained on large corpora. The comparison is achieved by using the
aforementioned models to bootstrap a neural network that learns to predict a
holistic score for scripts. Furthermore, we investigate augmenting our model
with error corrections and monitor the impact on performance. Our results show
that our error-oriented approach outperforms other comparable ones which is
further demonstrated when training on more data. Additionally, extending the
model with corrections provides further performance gains when data sparsity is
an issue.
