task_categories:text-generation task_categories:text2text-generation language:en language:zh license:apache-2.0 size_categories:1K<n<10K format:parquet modality:tabular modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
- config_name: all
  features:
  - name: instruction
    dtype: string
  - name: response
    dtype: string
  - name: lang_input
    dtype: string
  - name: lang_score_input
    dtype: float64
  - name: lang_output
    dtype: string
  - name: lang_score_output
    dtype: float64
  splits:
  - name: train
    num_bytes: 147899814
    num_examples: 6000
  download_size: 85354156
  dataset_size: 147899814
- config_name: default
  features:
  - name: instruction
    dtype: string
  - name: response
    dtype: string
  - name: lang_input
    dtype: string
  - name: lang_score_input
    dtype: float64
  - name: lang_output
    dtype: string
  - name: lang_score_output
    dtype: float64
  splits:
  - name: train
    num_bytes: 57557677.615
    num_examples: 2335
  download_size: 38901133
  dataset_size: 57557677.615
configs:
- config_name: all
  data_files:
  - split: train
    path: all/train-*
- config_name: default
  data_files:
  - split: train
    path: data/train-*
license: apache-2.0
task_categories:
- text-generation
- text2text-generation
language:
- en
- zh
size_categories:
- 1K<n<10K
---

# pszemraj/LongWriter-6k-reformat


`THUDM/LongWriter-6k` reformatted to dedicated columns. The default dataset config is english only, see the `"all"` config for other langs


GPT-4 tiktoken token count:

```
        token_count
count   2335.000000
mean    5295.221842
std     2771.696858
min        1.000000
25%     3556.000000
50%     4729.000000
75%     6266.000000
max    28436.000000
```


 - Total count:	12.36 M tokens