Learning Non-Autoregressive Models from Search for Unsupervised Sentence
  Summarization
  Text summarization aims to generate a short summary for an input text. In
this work, we propose a Non-Autoregressive Unsupervised Summarization (NAUS)
approach, which does not require parallel data for training. Our NAUS first
performs edit-based search towards a heuristically defined score, and generates
a summary as pseudo-groundtruth. Then, we train an encoder-only
non-autoregressive Transformer based on the search result. We also propose a
dynamic programming approach for length-control decoding, which is important
for the summarization task. Experiments on two datasets show that NAUS achieves
state-of-the-art performance for unsupervised summarization, yet largely
improving inference efficiency. Further, our algorithm is able to perform
explicit length-transfer summary generation.
