license:other size_categories:100M<n<1B format:json modality:text library:datasets library:dask library:mlcroissant arxiv:2101.00027 region:us
---
license: other
---

# Pile Uncopyrighted
In response to [authors demanding that LLMs stop using their works](https://tcrn.ch/3rtpIDn), here's a copy of [The Pile](https://huggingface.co/datasets/monology/pile) with all copyrighted content removed.  
Please consider using this dataset to train your future LLMs, to respect authors and abide by copyright law.  
Creating an uncopyrighted version of a larger dataset (ie RedPajama) is planned, with no ETA.  

**Methodology**  
Cleaning was performed by removing everything from the Books3, BookCorpus2, OpenSubtitles, YTSubtitles, and OWT2 subsets.  
Based on section 7.1 of [the original paper](https://arxiv.org/abs/2101.00027), these datasets are the only ones which are not explicitly allowed to be used in AI training.