SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic
  Auto-Encoders
  We propose an end-to-end neural model for zero-shot abstractive text
summarization of paragraphs, and introduce a benchmark task, ROCSumm, based on
ROCStories, a subset for which we collected human summaries. In this task,
five-sentence stories (paragraphs) are summarized with one sentence, using
human summaries only for evaluation. We show results for extractive and human
baselines to demonstrate a large abstractive gap in performance. Our model,
SummAE, consists of a denoising auto-encoder that embeds sentences and
paragraphs in a common space, from which either can be decoded. Summaries for
paragraphs are generated by decoding a sentence from the paragraph
representations. We find that traditional sequence-to-sequence auto-encoders
fail to produce good summaries and describe how specific architectural choices
and pre-training techniques can significantly improve performance,
outperforming extractive baselines. The data, training, evaluation code, and
best model weights are open-sourced.
