task_categories:question-answering task_categories:text-generation task_ids:open-domain-qa task_ids:abstractive-qa annotations_creators:machine-generated language_creators:machine-generated multilinguality:monolingual source_datasets:HuggingFaceFW/fineweb-edu language:en license:odc-by size_categories:10K<n<100K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us education finetuning question-answering
---
language:
- en
license: odc-by
annotations_creators:
- machine-generated
language_creators:
- machine-generated
language_creators_raw:
- found
multilinguality:
- monolingual
size_categories:
- 1K<n<10K
source_datasets:
- HuggingFaceFW/fineweb-edu
task_categories:
- question-answering
- text-generation
task_ids:
- open-domain-qa
- abstractive-qa
pretty_name: FineWeb-Edu Supervised Finetuning Dataset
tags:
- education
- finetuning
- question-answering
dataset_info:
  features:
    - name: input
      dtype: string
    - name: output
      dtype: string
---

# FineWeb-Edu Supervised Finetuning Dataset

## Model Description

This dataset is designed for training language models to generate supervised finetuning data from raw text. It consists of text passages and corresponding question-answer pairs in JSONLines format.

## Intended Use

The primary purpose of this dataset is to enable large language models (LLMs) to generate high-quality supervised finetuning data from raw text inputs, useful for creating custom datasets for specific domains or tasks.

## Training Data

- **Source**: HuggingFaceFW/fineweb-edu
- **Processing**: The first 10,000 entries were split into chunks of approximately 2048 characters each. Random entries were then selected for inference.
- **Generation Method**: 10-shot learning using a custom Llama 3.1 model
- **Support Set**: Learning samples generated using GPT4o mini

## Dataset Statistics

- **Total Entries**: 10,170
- **Train Set**: 8,136 entries
- **Test Set**: 2,034 entries

## Data Format

Each entry in the dataset contains:

- `input`: A text passage from the source dataset
- `output`: Generated question-answer pairs based on the input text
  - Each pair consists of a question ending with [QUESTION_END] then its corresponding answer ending with [ANSWER_END]

## Sample Entry

```json
{
  "input": "This codex originated in Santiago de Compostela, the most important pilgrim center in the Middle Ages and at the same time a meeting place of musicians with their various instruments...",
  "output": "Where did the codex originate, and why is it significant?[QUESTION_END]The codex originated in Santiago de Compostela, a significant pilgrim center and musical hub in the Middle Ages.[ANSWER_END]..."
}
```

## Limitations and Biases

- The dataset may inherit biases present in the source material and generated content.
- It focuses on educational content and may not represent all text types or domains.

## Ethical Considerations

Users should evaluate the generated question-answer pairs for accuracy and appropriateness before using them in downstream applications.

## Additional Information

- **License**: Refer to the [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) license for usage terms.
- **Version**: 1.0
- **Date Created**: August 27, 2024

