Machine Reading Comprehension: The Role of Contextualized Language
  Models and Beyond
  Machine reading comprehension (MRC) aims to teach machines to read and
comprehend human languages, which is a long-standing goal of natural language
processing (NLP). With the burst of deep neural networks and the evolution of
contextualized language models (CLMs), the research of MRC has experienced two
significant breakthroughs. MRC and CLM, as a phenomenon, have a great impact on
the NLP community. In this survey, we provide a comprehensive and comparative
review on MRC covering overall research topics about 1) the origin and
development of MRC and CLM, with a particular focus on the role of CLMs; 2) the
impact of MRC and CLM to the NLP community; 3) the definition, datasets, and
evaluation of MRC; 4) general MRC architecture and technical methods in the
view of two-stage Encoder-Decoder solving architecture from the insights of the
cognitive process of humans; 5) previous highlights, emerging topics, and our
empirical analysis, among which we especially focus on what works in different
periods of MRC researches. We propose a full-view categorization and new
taxonomies on these topics. The primary views we have arrived at are that 1)
MRC boosts the progress from language processing to understanding; 2) the rapid
improvement of MRC systems greatly benefits from the development of CLMs; 3)
the theme of MRC is gradually moving from shallow text matching to cognitive
reasoning.
