license:apache-2.0 size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us corpus-creator
---
license: apache-2.0
dataset_info:
  features:
  - name: ids
    dtype: string
  - name: texts
    dtype: string
  splits:
  - name: train
    num_bytes: 146361
    num_examples: 313
  download_size: 68506
  dataset_size: 146361
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
tags:
- corpus-creator
---
This dataset was created using [Corpus Creator](https://huggingface.co/spaces/davanstrien/corpus-creator). This dataset was created by parsing a corpus of text files into chunks of sentences using Llama Index.
        This processing was done with a chunk size of 256 and a chunk overlap of 0.