task_categories:summarization language:en size_categories:1K<n<10K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
task_categories:
- summarization
language:
- en
size_categories:
- 1K<n<10K
---
Original data from: https://github.com/armancohan/long-summarization

The first 3000 rows of the test split of the original dataset were processed and filtered as follows.

* In the original dataset, some sentences appear several times in the same article, even if they're only contained once in the original research paper.
For this reason, all dataset rows where the same sentence appeared more than once where removed.
* In the original dataset, every sentence is a separate string, and these strings are contained in arrays. Therefore, all these strings were merged in order to create one single string for each article body and one other for each article abstract.
* All line breaks were removed.
* Unnecessary spaces (e.g. before points or commas) were removed.
* The dataset was filteder by article and summary length.