MMM: Multi-stage Multi-task Learning for Multi-choice Reading
  Comprehension
  Machine Reading Comprehension (MRC) for question answering (QA), which aims
to answer a question given the relevant context passages, is an important way
to test the ability of intelligence systems to understand human language.
Multiple-Choice QA (MCQA) is one of the most difficult tasks in MRC because it
often requires more advanced reading comprehension skills such as logical
reasoning, summarization, and arithmetic operations, compared to the extractive
counterpart where answers are usually spans of text within given passages.
Moreover, most existing MCQA datasets are small in size, making the learning
task even harder. We introduce MMM, a Multi-stage Multi-task learning framework
for Multi-choice reading comprehension. Our method involves two sequential
stages: coarse-tuning stage using out-of-domain datasets and multi-task
learning stage using a larger in-domain dataset to help model generalize better
with limited data. Furthermore, we propose a novel multi-step attention network
(MAN) as the top-level classifier for this task. We demonstrate MMM
significantly advances the state-of-the-art on four representative MCQA
datasets.
