size_categories:1K<n<10K format:parquet library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: input_ids
    sequence: int32
  - name: attention_mask
    sequence: int8
  splits:
  - name: train
    num_bytes: 5441056.553482587
    num_examples: 1447
  - name: test
    num_bytes: 605397.4465174129
    num_examples: 161
  download_size: 1903411
  dataset_size: 6046454.0
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: test
    path: data/test-*
---

# Dataset Card for flan_v2_cot_fs_opt_train_cleaned

This dataset is a preprocessed subset of the FLAN V2 dataset, specifically derived from the "cot_fs_opt_train" task. It is intended for fine-tuning large language models like LLaMA2. Preprocessing steps included filtering out short responses (<100 tokens) and deduplication using cosine similarity.

## Dataset Details

### Dataset Description

A processed subset of FLAN V2 for LLaMA2 fine-tuning on the Chain of Thought (cot) task from "cot_fs_opt_train". Preprocessing involved filtering short responses and deduplication for improved quality.

- **Created by:** Fitsum Getachew Tola

- **Language(s) (NLP):** English

### Dataset Sources [optional]

- **Repository:** https://www.kaggle.com/datasets/lallucycle/flan-v2-part-1?select=cot_fs_opt_train.jsonl
## Uses

### Direct Use

Fine-tuning large language models for tasks requiring reasoning and following a Chain of Thought process.

### Out-of-Scope Use

- Tasks not requiring reasoning or Chain of Thought. 
- Training models on sensitive or private information if present.

## Dataset Structure

The dataset is in JSON Lines (JSONL) format, with each line representing a data point containing the following fields:

- `inputs`: The input prompt or question (text).
- `targets`: The corresponding response or answer (text).
- `task`: The task type, which is "cot" (Chain of Thought) in this subset.

## Dataset Creation

### Curation Rationale

This dataset was created to provide high-quality training data for fine-tuning large language models on reasoning tasks specifically in the Chain of Thought domain.

### Source Data

The source data is the FLAN V2 dataset for the "cot_fs_opt_train" task.

#### Data Collection and Processing

The original FLAN V2 data was filtered to remove entries with responses containing less than 100 tokens. Subsequently, deduplication was performed using TF-IDF and cosine similarity to eliminate highly similar responses (similarity > 0.95).

#### Who are the source data producers?

The source data producers are the creators of the FLAN V2 dataset (insert information here if known).


## Bias, Risks, and Limitations

- Potential biases inherited from the FLAN V2 dataset.
- Limited dataset size compared to the original FLAN V2.
- Focus on responses exceeding 100 tokens might exclude concise answers.

### Recommendations

Users should be aware of potential biases and limitations. Consider combining this data with other sources for a more comprehensive training dataset.


## Glossary [optional]

- FLAN: Few-Shot Learning for Natural Language Processing
- LLaMA: Pathways Language Model



## Dataset Card Authors

Fitsum Getachew Tola

## Dataset Card Contact

fitsumgeatachewtola@gmail.com