task_categories:text-classification language:en license:mit size_categories:100K<n<1M format:json modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2501.13007 region:us
---
license: mit
task_categories:
- text-classification
language:
- en
size_categories:
- 100K<n<1M
---

# PAIRJUDGE-432K Dataset

**PAIRJUDGE-432K** is a large-scale dataset containing 432K annotated pairwise judgments designed for training reward models in mathematical reasoning tasks. Each sample in the dataset is a prompt–completion pair where the prompt includes a math problem and two candidate solutions, and the completion is a chain-of-thought reasoning that evaluates the correctness of the two responses.

- Paper: https://arxiv.org/abs/2501.13007
- Code: https://github.com/THU-KEG/PairJudgeRM
- Model: https://huggingface.co/THU-KEG/PairJudge-432K


## Overview

- **Purpose:** To provide high-quality annotated pairwise comparisons for training models like PairJudge RM, enabling robust Best-of-N sampling through pairwise judgment.
- **Content:** 432K samples derived from the NumiaMath dataset and annotated with gemini-1.5-flash.
- **Format:** Each example is a prompt–completion pair:
  - **Prompt:** Contains a math problem *x* along with two candidate solutions (*y₁* and *y₂*).
  - **Completion:** A chain-of-thought reasoning text that verifies the correctness of each candidate and outputs corresponding correctness labels (*c₁* and *c₂*).

## Dataset Format & Structure

The dataset is organized into prompt–completion pairs that follow a consistent template:

- **Prompt Template:**  
  - **Question:** A math problem drawn from diverse sources (e.g., AMC/AIME, AoPS Forum, Chinese K-12, Olympiad problems).
  - **Responses:** Two candidate solutions generated by large language models.
  
- **Completion:**  
  - A detailed chain-of-thought reasoning process that:
    - Verifies each step of the candidate solutions.
    - Checks mathematical accuracy, logical consistency, and completeness.
    - Provides final correctness judgments for both responses.

## Construction Pipeline

1. **Data Collection:**  
   - Math problems were collected from the NumiaMath dataset, which includes problems from various educational and competitive sources.
   
2. **Candidate Generation:**  
   - For each math problem, multiple candidate solutions were generated (using models such as Llama-3.1-8B-Instruct) to provide a diverse set of responses.
   
3. **Annotation via Gemini-1.5-Flash:**  
   - The candidate solutions were then evaluated in pairwise settings using gemini-1.5-flash.  
   - A knockout tournament process was conducted to record pairwise judgments, where each match compares two candidate solutions.
   
4. **Filtering & Finalization:**  
   - Out of over 2.2M recorded comparisons, 1.3M high-quality judgments were retained.
   - Further filtering was applied to ensure adherence to the specified prompt template, resulting in the final 432K high-quality training samples.

## Dataset Statistics

The dataset aggregates problems from multiple sources. Although the original NumiaMath dataset includes nearly 860K problems, strict filtering for clarity, formatting, and response quality resulted in the final 432K examples. Key statistics include:

- **Total Samples:** 432K prompt–completion pairs.
- **Sources:** Problems from AMC/AIME, AoPS Forum, Chinese K-12, Olympiad, and other math problem datasets.
- **Annotations:** Each pair includes detailed chain-of-thought reasoning along with binary correctness labels for the two candidate responses.

## Usage

The PAIRJUDGE-432K dataset is ideal for:
- Training and fine-tuning reward models that perform pairwise comparisons.
- Enhancing Best-of-N sampling techniques in mathematical reasoning tasks.
- Research in model evaluation where transparent reasoning processes are required.

### Example Code

```python
from datasets import load_dataset

# Load the PAIRJUDGE-432K dataset from Hugging Face Datasets
dataset = load_dataset("THU-KEG/PairJudge-432K")

# View an example
print(dataset["train"][0])
```


## Citation
```
@article{liu2025PairJudge,
  title={PairJudge RM: Perform Best-of-N Sampling with Knockout Tournament},
  author={Liu, Yantao and Yao, Zijun and Min, Rui and Cao, Yixin and Hou, Lei and Li, Juanzi},
  journal={arXiv preprint arXiv:2501.13007},
  year={2025},
  note={in progress work},
  url={https://doi.org/10.48550/arXiv.2501.13007}
}
```


