task_categories:question-answering license:mit size_categories:10K<n<100K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: mit
task_categories:
- question-answering
---

This ML Q&A dataset contains 43,713 samples, where each includes three fields - question, context(title + abstract) and answer. 
It is created based on the original dataset aalksii/ml-arxiv-papers, which contains the titles and abstracts of ML ArXiv papers. 

To create question-answer pairs, the gpt-3.5-turbo API is called with the following prompt:  
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": f"Title: \"{title}\". Abstract: 
    \"{abstract}\". Given the above content, ask an AI-related question based on 
    the title and generate an answer based on the abstract in JSON format {"question":" 
    What/How xxx ?", "answer": "xxx." }. The question should be less than 25 tokens and 
    the answer between 100 to 200 tokens."}
]

After processing with the API, the low-quality Q&A samples are excluded. 


Good news: Llama-2-7B-Chat is already finetuned with this dataset!! And the checkpoint is available at https://huggingface.co/hanyueshf/llama-2-7b-chat-ml-qa.

Acknowledgement:  
Thank Xinyu Wang(王欣宇) and Linze Li(李林泽) for their efforts in creating this dataset.