Neural Language Models for Nineteenth-Century English
  We present four types of neural language models trained on a large historical
dataset of books in English, published between 1760-1900 and comprised of ~5.1
billion tokens. The language model architectures include static (word2vec and
fastText) and contextualized models (BERT and Flair). For each architecture, we
trained a model instance using the whole dataset. Additionally, we trained
separate instances on text published before 1850 for the two static models, and
four instances considering different time slices for BERT. Our models have
already been used in various downstream tasks where they consistently improved
performance. In this paper, we describe how the models have been created and
outline their reuse potential.
