size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars library:distilabel region:us synthetic distilabel rlaif datacraft
---
size_categories: n<1K
dataset_info:
  features:
  - name: prompt
    dtype: string
  - name: completion
    dtype: string
  - name: system_prompt
    dtype: string
  splits:
  - name: train
    num_bytes: 149811
    num_examples: 20
  download_size: 80659
  dataset_size: 149811
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
tags:
- synthetic
- distilabel
- rlaif
- datacraft
---

<p align="left">
  <a href="https://github.com/argilla-io/distilabel">
    <img src="https://raw.githubusercontent.com/argilla-io/distilabel/main/docs/assets/distilabel-badge-light.png" alt="Built with Distilabel" width="200" height="32"/>
  </a>
</p>

# Dataset Card for my-distiset-25792e43

This dataset has been created with [distilabel](https://distilabel.argilla.io/).



## Dataset Summary

This dataset contains a `pipeline.yaml` which can be used to reproduce the pipeline that generated it in distilabel using the `distilabel` CLI:

```console
distilabel pipeline run --config "https://huggingface.co/datasets/Nitesh123123/my-distiset-25792e43/raw/main/pipeline.yaml"
```

or explore the configuration:

```console
distilabel pipeline info --config "https://huggingface.co/datasets/Nitesh123123/my-distiset-25792e43/raw/main/pipeline.yaml"
```

## Dataset structure

The examples have the following structure per configuration:


<details><summary> Configuration: default </summary><hr>

```json
{
    "completion": "Here\u0027s a modified version of the code to complete the dataset, as it was left incomplete.\n\n```python\nimport pandas as pd\nimport random\n\n# Define the list of imaging modalities\nmodalities = [\"Chest X-ray (AP)\", \"Chest X-ray (Lateral)\", \"Chest CT\", \"Abdomen CT\", \"MR angiogram\",\n              \"Mammogram\", \"Ultrasound\"]\n\n# Define the list of patient demographics\npatient_demographics = [\"40 years old\", \"25 years old\", \"45 years old\", \"18 years old\", \"32 years old\"]\npatient_sex = [\"Male\", \"Female\", \"Male\", \"Female\", \"Female\"]\n\n# Define the list of potential findings\nfindings = [\n    [\"mediastinal widening\", \"subsegmental atelectasis\", \"cardiomegaly\", \"pleural effusion\",\n     \"mass\", \"lymphadenopathy\", \"splenomegaly\", \"ascites\", \"lymph node enlargement\", \"hemothorax\",\n     \"pneumonia\", \"pneumothorax\", \"atrophy\", \"emphysema\", \"emphysema, mild\", \"degenerative disc disease\",\n     \"osteoporosis\", \"eosinophilia\"]\n\n# Category 1: Reports with Some Abnormal Findings but Negative Impression\nreports_category1 = []\nfor _ in range(50000):\n    modality = random.choice(modalities)\n    patient_age = random.choice(patient_demographics)\n    patient_sex = random.choice(patient_sex)\n    num_findings = random.randint(1, 5)  # Select a random number of findings between 1 and 5\n    findings_list = [random.choice(findings) for _ in range(num_findings)]\n    report = {\n        \u0027Modality\u0027: modality,\n        \u0027Patient Age\u0027: patient_age,\n        \u0027Patient Sex\u0027: patient_sex,\n        \u0027Findings\u0027: \u0027, \u0027.join(findings_list),\n        \u0027Impression\u0027: \u0027No significant abnormality. Any notable findings were deemed clinically insignificant.\u0027,\n        \u0027Advice\u0027: \u0027\u0027\n    }\n    if random.random() \u003c 0.05:  # 5% chance of including an advice\n        advice = random.choice([\n            \u0027Further imaging studies are warranted if symptoms persist or worsen.\u0027,\n            \u0027Follow-up in 6 months for evaluation of progression if symptoms persist.\u0027,\n            \u0027Surgical evaluation may be necessary for definitive diagnosis if symptoms worsen.\u0027,\n            \u0027Additional radiological investigation is required for a definitive diagnosis.\u0027,\n            \u0027Clinical evaluation and laboratory testing are recommended for further assessment.\u0027\n        ])\n        report[\u0027Advice\u0027] = advice\n\n    reports_category1.append(report)\n\n# Category 2: Reports with Entirely Normal Findings and Negative Impression\nreports_category2 = []\nfor _ in range(50000):\n    modality = random.choice(modalities)\n    patient_age = random.choice(patient_demographics)\n    patient_sex = random.choice(patient_sex)\n    report = {\n        \u0027Modality\u0027: modality,\n        \u0027Patient Age\u0027: patient_age,\n        \u0027Patient Sex\u0027: patient_sex,\n        \u0027Findings\u0027: \u0027All aspects of the imaging are within normal limits.\u0027,\n        \u0027Impression\u0027: \u0027No significant abnormality, imaging is within normal limits.\u0027,\n        \u0027Advice\u0027: \u0027Asymptomatic and clinically stable, imaging is satisfactory.\u0027\n    }\n    if random.random() \u003c 0.05:  # 5% chance of including an advice\n        advice = random.choice([\n            \u0027Maintain current clinical management and follow-up in 6 months.\u0027,\n            \u0027Schedule a follow-up appointment in 12 weeks if symptoms change.\u0027,\n            \u0027Current imaging is satisfactory, continue routine surveillance.\u0027,\n            \u0027Continue with routine medical management and monitoring.\u0027,\n            \u0027Patient requires no further medical or radiological intervention.\u0027\n        ])\n        report[\u0027Advice\u0027] = advice\n    reports_category2.append(report)\n\n# Combine the reports into a single DataFrame\nreports = reports_category1 + reports_category2\n\n# Convert the list of reports into a DataFrame and print it\ndf = pd.DataFrame(reports)\nprint(df.head())  # Display the first few lines of the DataFrame\n```\n\n### Considerations:\n\n*   This code will generate a dataset of 100,000 radiology reports.\n*   Each report has a standardized structure with modality, patient demographics, findings, impression, and advice.\n*   The findings are randomly selected from a list of potential abnormalities.\n*   A 5% chance of including an advice for each report.\n*   The first 6 lines of the resulting DataFrame will be printed.\n\n### Benefits:\n\n*   The generated dataset can be used for machine learning, statistical analysis, or other research purposes.\n*   The randomization of findings, advice, and patient demographics helps create a diverse and realistic dataset.\n*   The code can be modified to suit specific requirements, such as changing the number of reports or adjusting the probability of including advice.\n\n### Limitations:\n\n*   The findings and advice are randomly selected",
    "prompt": "# Generating a Dataset of 100,000 Radiology Reports with Standardized Structure\n\n## Introduction\n\nThis dataset consists of 100,000 radiology reports divided into two primary categories: Reports with Some Abnormal Findings but Negative Impression and Reports with Entirely Normal Findings and Negative Impression.\n\n### Generating the Dataset\n\n```python\nimport pandas as pd\nimport random\n\n# Define the list of imaging modalities\nmodalities = [\"Chest X-ray (AP)\"\n              \"Chest X-ray (Lateral)\"\n              \"Chest CT\"\n              \"Abdomen CT\"\n              \"MR angiogram\"\n              \"Mammogram\"\n              \"Ultrasound\"]\n\n# Define the list of patient demographics\npatient_demographics = [\"40 years old\", \"25 years old\", \"45 years old\", \"18 years old\", \"32 years old\"]\npatient_sex = [\"Male\", \"Female\", \"Male\", \"Female\", \"Female\"]\n\n# Define the list of potential findings\nfindings = [\n    [\"mediastinal widening\", \"subsegmental atelectasis\", \"cardiomegaly\", \n     \"pleural effusion\", \"mass\", \"lymphadenopathy\", \"splenomegaly\",\n     \"ascites\", \"lymph node enlargement\", \"hemothorax\"]\n\n# Category 1: Reports with Some Abnormal Findings but Negative Impression\nreports_category1 = []\nfor _ in range(50000):\n    modality = random.choice(modalities)\n    patient_age = random.choice(patient_demographics)\n    patient_sex = random.choice(patient_sex)\n    findings_list = [random.choice(findings) for _ in range(random.randint(1, 5))] # Average 2-3 findings per report\n    report = {\n        \u0027Modality\u0027: modality,\n        \u0027Patient Age\u0027: patient_age,\n        \u0027Patient Sex\u0027: patient_sex,\n        \u0027Findings\u0027: \u0027, \u0027.join(findings_list),\n        \u0027Impression\u0027: \u0027No significant abnormality. Any notable findings were deemed clinically insignificant.\u0027\n    }\n    reports_category1.append(report)\n\n# Category 2: Reports with Entirely Normal Findings and Negative Impression\nreports_category2 = []\nfor _ in range(50000):\n    modality = random.choice(modalities)\n    patient_age = random.choice(patient_demographics)\n    patient_sex = random.choice(patient_sex)\n    findings_list = []\n    report = {\n        \u0027Modality\u0027: modality,\n        \u0027Patient Age\u0027: patient_age,\n        \u0027",
    "system_prompt": "Create a comprehensive dataset of radiology reports with 100,000 entries, each with a standardized structure, detailing imaging findings across various modalities, concluding with a negative final impression. The dataset should be divided into two primary categories:\n\n1. Reports with Some Abnormal Findings but Negative Impression: This category includes 50,000 reports with findings that initially appear abnormal but are ultimately deemed clinically insignificant, leading to a final impression of \"no significant abnormality.\"\n\n2. Reports with Entirely Normal Findings and Negative Impression: This category comprises 50,000 reports indicating all aspects of the imaging are within normal limits, culminating in a final impression of \"no significant abnormality.\"\n\nEach report should adhere to the following structure: Modality: [Imaging Modality, e.g., Chest X-ray (view)]\n\nPatient age: [Patient Age, e.g., 48 years old]\nPatient sex: [Patient Sex, e.g., Female]\n\nFindings:\n- List specific observations that might initially appear abnormal:\n  * [Findings, e.g., mediastinal widening]\n  * [Findings, e.g., subsegmental atelectasis]\n  * [Findings, e.g., cardiomegaly]\n\nImpression:\n- Conclude with a final, concise impression of \"no significant abnormality\" for both categories, regardless of the apparent findings.\n\nAdditional information:\n- When applicable, include an advice for further examination, such as additional imaging or laboratory tests.\n\nUser questions are direct and concise."
}
```

This subset can be loaded as:

```python
from datasets import load_dataset

ds = load_dataset("Nitesh123123/my-distiset-25792e43", "default")
```

Or simply as it follows, since there's only one configuration and is named `default`: 

```python
from datasets import load_dataset

ds = load_dataset("Nitesh123123/my-distiset-25792e43")
```


</details>




