task_categories:summarization language:de size_categories:1K<n<10K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
task_categories:
- summarization
language:
- de
size_categories:
- 1K<n<10K
---

Train split of german MLSUM filtered by using the code by Aumiller et al. (1) available at https://github.com/dennlinger/summaries/tree/main with following settings:

min_length_summary=15, min_length_reference=200, length_metric="whitespace", extractiveness="fully"

After this filtering operation, the dataset was filtered again to exclude all samples with bigram overlap fraction between summary and article >= 0,76.

Finally, 10k random saples were selected and kept in this dataset. All other samples were removed.

(1): Aumiller, D., Fan, J., & Gertz, M. (2023). On the State of German (Abstractive) Text Summarization. arXiv preprint arXiv:2301.07095.