license:bsd size_categories:10K<n<100K format:json modality:text library:datasets library:dask library:mlcroissant arxiv:2405.11647 region:us
---
license: bsd
---
# Dataset Card for Hummer

## Dataset Summary


This repository provides access to data sampled, cleaned, and categorized by dimensions from pair-wise data built from original UltraFeedback dataset, as presented in [Hummer: Towards Limited Competitive Preference Dataset](https://arxiv.org/abs/2405.11647).These data are meant to train preference (or reward) models for subsequent and balanced RLHF training. These data are *not* meant for supervised training of dialogue agents. Training dialogue agents on these data is likely to lead to harmful models and this shold be avoided.

**Disclaimer**: The data (especially the accuracy alignment dimension data) contain content that has not been examined to 100% get rid of harmful paragraphs. Topics may include, but are not limited to, discriminatory language and discussions of abuse, violence, self-harm, exploitation, and other potentially upsetting subject matter. Please only engage with the data in accordance with your own personal risk tolerance. The data are intended for research purposes, especially research that can make models *less* harmful. The views expressed in the data do not reflect the views of Anthropic or any of its employees. As mentioned above, these data are *not* intended for training dialogue agents as this will likely lead to harmful model behavior. 

Details of the dataset are described further below. 

## Hummer data

The data are described in the paper: [Hummer: Towards Limited Competitive Preference Dataset](https://arxiv.org/abs/2405.11647). If you find the data useful, please cite the paper. The data format is very simple -- each line of the jsonl files contains a triplet of texts,one "prompt", one "chosen" and one "rejected".

The data is categorized into 6 different individual dimensions.

Details about the data collection, finegraining, abstracting, and categorizing process can be found in the paper, specifically in Section 4,6 and Appendix B. 

Each line of the JSONL file contains a dictionary with the following fields:
- `prompt` the text of the question one human asks one language model to respond
- `chosen` a more favored response the language model gives in response to 'prompt', under the guidance of dimension of certain sub-dataset
- `rejected` a less favored or disliked response the language model gives in response to 'prompt', under the guidance of dimension of certain sub-dataset


## Usage

Each of the above sub-datasets of different alignment dimensions is located in a separate sub-directory. To load an individual subset, use the `data_dir` argument of the `load_dataset()` function as follows:

```python
from datasets import load_dataset
# Load all subsets of different dimensions (share the same schema)
dataset = load_dataset("sarinw-2024/hummer")
# Load one of the subsets, for a certain dimension
dataset = load_dataset("sarinw-2024/hummer", data_dir="accuracy")
```
## Citation
```
@article{jiang2024hummer,
  title={Hummer: Towards limited competitive preference dataset},
  author={Jiang, Li and Wu, Yusen and Xiong, Junwu and Ruan, Jingqing and Ding, Yichuan and Guo, Qingpei and Wen, Zujie and Zhou, Jun and Deng, Xiaotie},
  journal={Conference on Language Modeling},
  year={2024}
}
```
## Contact

You can submit inquiries to: {sarinw2023, jiangli3859}@gmail.com