task_categories:text-generation source_datasets:HuggingFaceH4/MATH-500 language:en license:mit size_categories:n<1K format:csv modality:tabular modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: mit
task_categories:
- text-generation
language:
- en
pretty_name: MATH-500
size_categories:
- n<1K
size: 20
source_datasets:
  - HuggingFaceH4/MATH-500
model_usage:
  - name: Qwen/Qwen2.5-1.5B-Instruct
    purpose: Generating solutions to math problems via chain-of-thought reasoning.
  - name: Skywork/Skywork-o1-Open-PRM-Qwen-2.5-1.5B
    purpose: Scoring step-by-step reasoning solutions with a process reward model.
problem_levels:
  - levels_included: [1, 2, 3]
    description: Problems categorized as easier to moderate difficulty, suitable for evaluation with smaller models.
---
# Dataset Card for Dataset Name

This dataset contains 20 mathematical problems selected from the MATH-500 dataset, focusing on levels 1–3. 
It includes solutions generated by "Qwen/Qwen2.5-1.5B-Instruct" using two methods:
  1. Greedy decoding with Chain-of-Thought prompting.
  2. Best-of-N sampling with Chain-of-Thought prompting with weighted selection (N=16). The selection is obtained using the last reward from "Skywork/Skywork-o1-Open-PRM-Qwen-2.5-1.5B".
The dataset is designed to evaluate and compare the performance of these solution generation methods. 
Each problem includes the original question, solutions generated by the models, their associated reward scores, and whether the greedy answers and Best-of-N answers are correct.

## Dataset Details

### Dataset Description

This dataset was created as part of an exercise to explore the effectiveness of Best-of-N sampling and weighted selection in improving language model performance on mathematical reasoning problems. It only includes problems that are rated as level 1, 2 and 3. It leverages both generative models and reward models for evaluation and comparison.

- **Curated by:** Sören Dréano
- **Language(s) (NLP):** English
- **License:** MIT (identical to the original PRM800K dataset)

### Usage:

- Compute accuracy of different solution generation strategies.
- Compare performance between greedy decoding and Best-of-N weighted selection.
- Evaluate the effectiveness of reward models for improving reasoning quality.

### Columns:

- "problem": string, the problem to solve
- "solution": string, the solution to the problem
- "answer": string, the content of the \boxed{answer} in the solution
- "subject": string, the mathematical category of the problem
- "level": int, the level of difficulty of the problem (1 to 3)
- "unique_id": string, the source of the data
- "greedy_response": string, the greedy solution from Qwen2
- "greedy_result": string, the content of the \boxed{answer} in greedy_response
- "diverse_response": list[string], the solutions from Qwen2 with num_return_sequences=16
- "diverse_result": list[string], the contents of the \boxed{answer} in diverse_response
- "greedy_last_reward": float, the last reward given by Open-PRM to greedy_response
- "diverse_last_rewards": list[float], the last rewards given by Open-PRM to diverse_response*
- "greedy_accuracy": bool, whether greedy_result matches with answer
- "best_of_n_result": string, the content of the \boxed{answer} in diverse_result with the best reward
- "greedy_accuracy": bool, whether greedy_result matches with answer
- "best_of_n_accuracy": bool, whether best_of_n_result matches with answer

### Dataset Sources

- **MATH-500:** https://huggingface.co/datasets/HuggingFaceH4/MATH-500
- **PRM800k:** https://github.com/openai/prm800k

### Models

- **Problem Solving:** https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct
- **Rewards:** https://huggingface.co/Skywork/Skywork-o1-Open-PRM-Qwen-2.5-1.5B

### Reproduction

The code to reproduce this dataset is fully available: https://github.com/SorenDreano/MATH-500-subset-qwen2-answers-evaluated-by-open-PRM

## Dataset Card Authors 

Sören Dréano