size_categories:100K<n<1M format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: conversations
    list:
    - name: from
      dtype: string
    - name: value
      dtype: string
  splits:
  - name: train
    num_bytes: 78494888
    num_examples: 105851
  download_size: 31275284
  dataset_size: 78494888
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---

# synthetic_text_to_sql

ShareGPT version of [gretelai/synthetic_text_to_sql](https://huggingface.co/datasets/gretelai/synthetic_text_to_sql) using the following code:

```python
from datasets import load_dataset, DatasetDict

# Load the dataset
dataset = load_dataset('gretelai/synthetic_text_to_sql', split='all')

def format_sample(sample):
    conversations = [
        {
            "from": "human",
            "value": f"{sample['sql_context']}\n\n{sample['sql_prompt']}"
        },
        {
            "from": "gpt",
            "value": f"{sample['sql']}\n\n{sample['sql_explanation']}"
        }
    ]
    return {"conversations": conversations}

dataset = dataset.map(format_sample, remove_columns=dataset.column_names)
dataset = DatasetDict({'train': dataset})
```

It means that the `sql_context` and `sql_prompt` fields are concatenated as user instruction, and the `sql` and `sql_explanation` are concatenated as answers.
Ideally, we'd want to steer the model's answer by providing the explanations first.
However, they're not phrased in a way that would make sense if they appeared before the code, which is why I decided to append them after the `sql` field.

Let me know if you think that another formatting would be better.