size_categories:10K<n<100K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: text
    dtype: string
  - name: len_tokens
    dtype: int64
  splits:
  - name: train
    num_bytes: 44143916
    num_examples: 48016
  download_size: 8526144
  dataset_size: 44143916
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---

#tokenizer: google/gemma-2b-it

# hist len_tokens
![image/png](https://cdn-uploads.huggingface.co/production/uploads/641b435ba5f876fe30c5ae0a/X2A-k5fBWvLYFo64_X2P_.png)

```
<bos><start_of_turn>system
You are a helpful AI assistant.
solo responde en formato json.
lista de codigos linguisticos disponibles: ["es", "en", "fr", "de"].<end_of_turn>
<start_of_turn>user
{ "input": "es",
  "targets": "fr",
  "inputs_es": "¿Qué presidente de los Estados Unidos nunca se ha casado?"
}<end_of_turn>
<start_of_turn>model
{ "targets": "fr", 
  "targets_fr": "James Buchanan est le seul président qui ne s'est jamais marié."
}<end_of_turn><eos>
```

# describe.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/641b435ba5f876fe30c5ae0a/I_PhchRaeuCbODzw-fOFW.png)

# percentil.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/641b435ba5f876fe30c5ae0a/hGNK_4Dv5gLW1sQrcCZT1.png)

