End-to-end Named Entity Recognition and Relation Extraction using
  Pre-trained Language Models
  Named entity recognition (NER) and relation extraction (RE) are two important
tasks in information extraction and retrieval (IE \& IR). Recent work has
demonstrated that it is beneficial to learn these tasks jointly, which avoids
the propagation of error inherent in pipeline-based systems and improves
performance. However, state-of-the-art joint models typically rely on external
natural language processing (NLP) tools, such as dependency parsers, limiting
their usefulness to domains (e.g. news) where those tools perform well. The few
neural, end-to-end models that have been proposed are trained almost completely
from scratch. In this paper, we propose a neural, end-to-end model for jointly
extracting entities and their relations which does not rely on external NLP
tools and which integrates a large, pre-trained language model. Because the
bulk of our model's parameters are pre-trained and we eschew recurrence for
self-attention, our model is fast to train. On 5 datasets across 3 domains, our
model matches or exceeds state-of-the-art performance, sometimes by a large
margin.
