size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: filename
    dtype: string
  - name: chunks
    dtype: string
  splits:
  - name: train
    num_bytes: 139173
    num_examples: 303
  download_size: 50618
  dataset_size: 139173
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---

# Dataset info

This dataset contains chunks of the documentation of `argilla-io/argilla-python` repository, as obtained running the script [docs_dataset.py](https://github.com/argilla-io/distilabel-workbench/blob/agus/projects/argilla-sdk-bot/docs_dataset.py) like:

```bash
python docs_dataset.py \
    "argilla-io/argilla-python" \
    "plaguss/argilla_sdk_docs_raw" \
    --max_chars 512
```

Ideally it could be used to start a knowledge base from your documentation.

## Disclaimer

The script to generate the dataset is still experimental:

- There are repeated chunks (definitely a bug).
- The chunks generated are not properly sorted (although not entirely relevant).
- The code should be partitioned better (currently there's no distinction between code/plain text).


