source_datasets:starfishmedical/webGPT_x_dolly license:cc-by-sa-3.0 size_categories:10K<n<100K format:csv modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: cc-by-sa-3.0
source_datasets:
  - starfishmedical/webGPT_x_dolly
---

Conversion of [starfishmedical/webGPT_x_dolly](https://huggingface.co/datasets/starfishmedical/webGPT_x_dolly) dataset to be used in pretraining.

Python code used for conversion:

```python
from datasets import load_dataset
import pandas

dataset = load_dataset("starfishmedical/webGPT_x_dolly", split="train")

def format(columns):
    question = columns["instruction"].strip()
    answer = columns["output"].strip()
    return f"{question}\n\n{answer}"

pandas.DataFrame({"text": [format(columns) for columns in dataset]}).to_csv("train.csv", index=False)
```
