task_categories:question-answering task_categories:multiple-choice language:en license:apache-2.0 size_categories:100K<n<1M format:json modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2405.01886 region:us medical biology
---
license: apache-2.0
task_categories:
- question-answering
- multiple-choice
language:
- en
tags:
- medical
- biology
size_categories:
- 100K<n<1M
---

# Dataset Card for medmcqa-cot

<!-- Provide a quick summary of the dataset. -->

Synthetically enhanced responses to the medmcqa dataset using mixtral.

## Dataset Details

### Dataset Description

<!-- Provide a longer summary of what this dataset is. -->

To increase the quality of answers from the training splits of the [MedMCQA](https://medmcqa.github.io/) dataset, we leverage Mixtral-8x7B to generate Chain of Thought(CoT) answers. We create a custom prompt for the dataset, along with a
hand-crafted list of few-shot examples. For a multichoice answer, we ask the model to rephrase and explain the question, then explain each option with respect to the question, then summarise this explanation to arrive at the final solution. During this synthetic data generation process, the model is also given the solution
and the reference answer. For the cases where the model fails to generate correct responses and just reiterates the input question, we regenerate the solutions until a correct response is generated. More details are available in the paper.

- **Curated by:** [Ashwin Kumar Gururajan](https://huggingface.co/G-AshwinKumar)
- **Language(s) (NLP):** English
- **License:** Apache 2.0

### Dataset Sources

<!-- Provide the basic links for the dataset. -->

- **Paper:** [Aloe: A Family of Fine-tuned Open Healthcare LLMs](https://arxiv.org/abs/2405.01886)

## Dataset Creation

### Curation Rationale

This dataset was created to provide a high quality easy to use instruction tuning dataset based on medmcqa.

## Citation

<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**
```
@misc{gururajan2024aloe,
      title={Aloe: A Family of Fine-tuned Open Healthcare LLMs}, 
      author={Ashwin Kumar Gururajan and Enrique Lopez-Cuena and Jordi Bayarri-Planas and Adrian Tormos and Daniel Hinjos and Pablo Bernabeu-Perez and Anna Arias-Duart and Pablo Agustin Martin-Torres and Lucia Urcelay-Ganzabal and Marta Gonzalez-Mallo and Sergio Alvarez-Napagao and Eduard Ayguadé-Parra and Ulises Cortés Dario Garcia-Gasulla},
      year={2024},
      eprint={2405.01886},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

@InProceedings{pmlr-v174-pal22a,
  title = 	 {MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering},
  author =       {Pal, Ankit and Umapathi, Logesh Kumar and Sankarasubbu, Malaikannan},
  booktitle = 	 {Proceedings of the Conference on Health, Inference, and Learning},
  pages = 	 {248--260},
  year = 	 {2022},
  editor = 	 {Flores, Gerardo and Chen, George H and Pollard, Tom and Ho, Joyce C and Naumann, Tristan},
  volume = 	 {174},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {07--08 Apr},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v174/pal22a/pal22a.pdf},
  url = 	 {https://proceedings.mlr.press/v174/pal22a.html},
  abstract = 	 {This paper introduces MedMCQA, a new large-scale, Multiple-Choice Question Answering (MCQA) dataset designed to address real-world medical entrance exam questions. More than 194k high-quality AIIMS & NEET PG entrance exam MCQs covering 2.4k healthcare topics and 21 medical subjects are collected with an average token length of 12.77 and high topical diversity. Each sample contains a question, correct answer(s), and other options which requires a deeper language understanding as it tests the 10+ reasoning abilities of a model across a wide range of medical subjects & topics. A detailed explanation of the solution, along with the above information, is provided in this study.}
}
```

## Dataset Card Authors

[Ashwin Kumar Gururajan](https://huggingface.co/G-AshwinKumar)

## Dataset Card Contact

[hpai@bsc.es](mailto:hpai@bsc.es)