language:en license:cc-by-4.0 size_categories:1K<n<10K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2406.08673 region:us human-feedback
---
license: cc-by-4.0
dataset_info:
  features:
  - name: prompt
    dtype: string
  - name: chosen_response
    dtype: string
  - name: rejected_response
    dtype: string
  splits:
  - name: train
    num_bytes: 26707924
    num_examples: 7221
  - name: validation
    num_bytes: 1369079
    num_examples: 373
  download_size: 14402663
  dataset_size: 28077003
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: validation
    path: data/validation-*
language:
- en
tags:
- human-feedback
pretty_name: HelpSteer2-DPO
size_categories:
- 1K<n<10K
---

This dataset is a conversion of [nvidia/HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) into preference pairs based on the helpfulness score for training DPO.

HelpSteer2-DPO is also licensed under CC-BY-4.0.

## Dataset Description
In accordance with the following paper, [HelpSteer2: Open-source dataset for training top-performing reward models](http://arxiv.org/abs/2406.08673)

we converted [nvidia/HelpSteer2](https://huggingface.co/datasets/nvidia/HelpSteer2) dataset into a preference dataset by taking the response with the higher helpfulness score as the chosen response, with the remaining response being the rejected response. In cases where the helpfulness scores were identical, we discarded that pair entirely. 
This became our HelpSteer2 DPO dataset, which contains 7,221 training samples and 373 validation samples.


Last Update: 2024-07-11