Domain Adaptation with Pre-trained Transformers for Query Focused
  Abstractive Text Summarization
  The Query Focused Text Summarization (QFTS) task aims at building systems
that generate the summary of the text document(s) based on the given query. A
key challenge in addressing this task is the lack of large labeled data for
training the summarization model. In this paper, we address this challenge by
exploring a series of domain adaptation techniques. Given the recent success of
pre-trained transformer models in a wide range of natural language processing
tasks, we utilize such models to generate abstractive summaries for the QFTS
task for both single-document and multi-document scenarios. For domain
adaptation, we apply a variety of techniques using pre-trained
transformer-based summarization models including transfer learning, weakly
supervised learning, and distant supervision. Extensive experiments on six
datasets show that our proposed approach is very effective in generating
abstractive summaries for the QFTS task while setting a new state-of-the-art
result in several datasets across a set of automatic and human evaluation
metrics.
