license:mit region:us
---
license: mit
---

Modification of the [newsroom](https://huggingface.co/datasets/newsroom) dataset in Hugging Face. The main goal is to reproduce the results on BART. 

References: https://huggingface.co/datasets/newsroom/blob/main/newsroom.py

Major changes: 
1. replace the "\n\n" with one whitespace in the text 
2. remove the html related tags, remove the latin1 coded characters in the summary.
3. remove unnessary data features.

The main motivation for doing such modifications is that with the original data, particularly the summary data, the prediction will be weird and would involve multiple lines, which then makes the number of predictions unmatched with the number of references. 