size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars library:distilabel arxiv:2312.08935 region:us synthetic distilabel rlaif
---
size_categories: n<1K
dataset_info:
  features:
  - name: instruction
    dtype: string
  - name: answer
    dtype: string
  - name: golden_solution
    dtype: string
  - name: model_name
    dtype: 'null'
  - name: solutions
    dtype: string
  - name: distilabel_metadata
    struct:
    - name: raw_input_completer
      dtype: 'null'
    - name: raw_input_generator
      list:
      - name: content
        dtype: string
      - name: role
        dtype: string
    - name: raw_input_golden_generator
      list:
      - name: content
        dtype: string
      - name: role
        dtype: string
    - name: raw_output_completer
      dtype: 'null'
    - name: raw_output_generator
      dtype: string
    - name: raw_output_golden_generator
      dtype: string
  - name: completions
    dtype: 'null'
  splits:
  - name: train
    num_bytes: 32163
    num_examples: 3
  download_size: 67383
  dataset_size: 32163
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
tags:
- synthetic
- distilabel
- rlaif
---

<p align="left">
  <a href="https://github.com/argilla-io/distilabel">
    <img src="https://raw.githubusercontent.com/argilla-io/distilabel/main/docs/assets/distilabel-badge-light.png" alt="Built with Distilabel" width="200" height="32"/>
  </a>
</p>

# Dataset Card for test_math_shepherd_v3

This dataset has been created with [distilabel](https://distilabel.argilla.io/).



## Dataset Summary

This dataset contains a `pipeline.yaml` which can be used to reproduce the pipeline that generated it in distilabel using the `distilabel` CLI:

```console
distilabel pipeline run --config "https://huggingface.co/datasets/plaguss/test_math_shepherd_v3/raw/main/pipeline.yaml"
```

or explore the configuration:

```console
distilabel pipeline info --config "https://huggingface.co/datasets/plaguss/test_math_shepherd_v3/raw/main/pipeline.yaml"
```

## Dataset structure

The examples have the following structure per configuration:


<details><summary> Configuration: default </summary><hr>

```json
{
    "answer": "Janet sells 16 - 3 - 4 = \u003c\u003c16-3-4=9\u003e\u003e9 duck eggs a day.\nShe makes 9 * 2 = $\u003c\u003c9*2=18\u003e\u003e18 every day at the farmer\u2019s market.\n#### 18",
    "completions": null,
    "distilabel_metadata": {
        "raw_input_completer": null,
        "raw_input_generator": [
            {
                "content": "You are a math tutor that helps students solve math problems by breaking them down into clear, logical steps. Follow these guidelines:\n\n# For each step:\n- Clearly explain the reasoning\n- Show the calculated result for any arithmetic calculation\n- Present intermediate calculations clearly\n- Use clear, concise language to explain the mathematical reasoning\n\n# Format requirements:\n- Number each step starting with \"Step 1:\"\n- The final step should clearly state \"The answer is: [result]\"\n- Keep explanations clear and concise\n\n# Rules:\n- All calculations must be shown within \u003c\u003c\u003e\u003e brackets\n- Basic operations: use * for multiplication, / for division, + for addition, - for subtraction\n- Write the full calculation and result, e.g., \u003c\u003c5*10=50\u003e\u003e50\n\n# Examples:\n## Input\nA store sells notebooks for $3 each. If you buy 5 or more, you get a 20% discount. How much would you pay for 6 notebooks?\n\n## Output\nStep 1: Calculate the regular price for 6 notebooks: 6 * $3 = \u003c\u003c63=18\u003e\u003e18 dollars\nStep 2: Calculate the 20% discount amount: 18 * 20/100 = \u003c\u003c1820/100=3.6\u003e\u003e3.6 dollars\nStep 3: Subtract the discount from the regular price: 18 - 3.6 = \u003c\u003c18-3.6=14.4\u003e\u003e14.4 dollars. The answer is: 14.4\n\n## Input\nA recipe calls for 2.5 cups of flour to make 12 cookies. How many cups of flour are needed to make 30 cookies?\n\n## Output\nStep 1: Find out how many cups of flour are needed per cookie: 2.5 \u00f7 12 = \u003c\u003c2.5/12=0.208333\u003e\u003e0.208333 cups\nStep 2: Calculate the flour needed for 30 cookies: 0.208333 * 30 = \u003c\u003c0.208333*30=6.25\u003e\u003e6.25 cups. The answer is: 6.25\n",
                "role": "system"
            },
            {
                "content": "Generate 5 example solutions to the following problem, separated by a single `---`:\nJanet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers\u0027 market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers\u0027 market?",
                "role": "user"
            }
        ],
        "raw_input_golden_generator": [
            {
                "content": "You are a math tutor that helps students solve math problems by breaking them down into clear, logical steps. Follow these guidelines:\n\n# For each step:\n- Clearly explain the reasoning\n- Show the calculated result for any arithmetic calculation\n- Present intermediate calculations clearly\n- Use clear, concise language to explain the mathematical reasoning\n\n# Format requirements:\n- Number each step starting with \"Step 1:\"\n- The final step should clearly state \"The answer is: [result]\"\n- Keep explanations clear and concise\n\n# Rules:\n- All calculations must be shown within \u003c\u003c\u003e\u003e brackets\n- Basic operations: use * for multiplication, / for division, + for addition, - for subtraction\n- Write the full calculation and result, e.g., \u003c\u003c5*10=50\u003e\u003e50\n\n# Examples:\n## Input\nA store sells notebooks for $3 each. If you buy 5 or more, you get a 20% discount. How much would you pay for 6 notebooks?\n\n## Output\nStep 1: Calculate the regular price for 6 notebooks: 6 * $3 = \u003c\u003c63=18\u003e\u003e18 dollars\nStep 2: Calculate the 20% discount amount: 18 * 20/100 = \u003c\u003c1820/100=3.6\u003e\u003e3.6 dollars\nStep 3: Subtract the discount from the regular price: 18 - 3.6 = \u003c\u003c18-3.6=14.4\u003e\u003e14.4 dollars. The answer is: 14.4\n\n## Input\nA recipe calls for 2.5 cups of flour to make 12 cookies. How many cups of flour are needed to make 30 cookies?\n\n## Output\nStep 1: Find out how many cups of flour are needed per cookie: 2.5 \u00f7 12 = \u003c\u003c2.5/12=0.208333\u003e\u003e0.208333 cups\nStep 2: Calculate the flour needed for 30 cookies: 0.208333 * 30 = \u003c\u003c0.208333*30=6.25\u003e\u003e6.25 cups. The answer is: 6.25\n",
                "role": "system"
            },
            {
                "content": "\nJanet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers\u0027 market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers\u0027 market?",
                "role": "user"
            }
        ],
        "raw_output_completer": null,
        "raw_output_generator": "### Example 1\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\nStep 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\nStep 4: Calculate the amount of money Janet makes from selling eggs: $0\n\n### Example 2\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\nStep 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\nStep 4: Calculate the amount of money Janet makes from selling eggs: $0\n\n### Example 3\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\nStep 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\nStep 4: Calculate the amount of money Janet makes from selling eggs: $0\n\n### Example 4\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\nStep 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\nStep 4: Calculate the amount of money Janet makes from selling eggs: $0\n\n### Example 5\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\nStep 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\nStep 4: Calculate the amount of money Janet makes from selling eggs: $0\n\n---\n\n### Example 6\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\nStep 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\nStep 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars. The answer is: 18\n\n### Example 7\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\nStep 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\nStep 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars. The answer is: 18\n\n### Example 8\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\nStep 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\nStep 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars. The answer is: 18\n\n### Example 9\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\nStep 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\nStep 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars. The answer is: 18",
        "raw_output_golden_generator": "Step 1: Calculate the total number of eggs laid by Janet\u0027s ducks per day: 16 eggs/day\n\nStep 2: Calculate the number of eggs Janet eats for breakfast: 3 eggs\nStep 3: Calculate the number of eggs Janet bakes: 4 eggs\nStep 4: Calculate the total number of eggs used (eaten or baked) per day: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\n\nStep 5: Calculate the number of eggs left for the farmers\u0027 market: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\n\nStep 6: Calculate the amount of money Janet makes from selling the eggs at the farmers\u0027 market: 9 eggs * $2/egg = \u003c\u003c9*2=18\u003e\u003e18 dollars\n\nThe answer is: 18"
    },
    "golden_solution": "[\"Step 1: Calculate the total number of eggs laid by Janet\u0027s ducks per day: 16 eggs/day\", \"Step 2: Calculate the number of eggs Janet eats for breakfast: 3 eggs\", \"Step 3: Calculate the number of eggs Janet bakes: 4 eggs\", \"Step 4: Calculate the total number of eggs used (eaten or baked) per day: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\", \"Step 5: Calculate the number of eggs left for the farmers\u0027 market: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\", \"Step 6: Calculate the amount of money Janet makes from selling the eggs at the farmers\u0027 market: 9 eggs * $2/egg = \u003c\u003c9*2=18\u003e\u003e18 dollars\", \"The answer is: 18\"]",
    "instruction": "Janet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers\u0027 market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers\u0027 market?",
    "model_name": null,
    "solutions": "[[\"Step 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs +\", \"Step 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs +\", \"Step 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day. +\", \"Step 4: Calculate the amount of money Janet makes from selling eggs: $0\\n\\n### Example 2\", \"Step 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\", \"Step 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\", \"Step 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\", \"Step 4: Calculate the amount of money Janet makes from selling eggs: $0\\n\\n### Example 3\", \"Step 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\", \"Step 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\", \"Step 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\", \"Step 4: Calculate the amount of money Janet makes from selling eggs: $0\\n\\n### Example 4\", \"Step 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\", \"Step 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\", \"Step 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\", \"Step 4: Calculate the amount of money Janet makes from selling eggs: $0\\n\\n### Example 5\", \"Step 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\", \"Step 2: Calculate the number of eggs Janet eats and bakes: 16 + 3 + 4 = \u003c\u003c16+3+4=23\u003e\u003e23 eggs\", \"Step 3: Calculate the remaining eggs: 16 - 23 = \u003c\u003c16-23=-7\u003e\u003e-7 eggs. Since we can\u0027t sell a negative number of eggs, we\u0027ll assume Janet doesn\u0027t sell any eggs on this day.\", \"Step 4: Calculate the amount of money Janet makes from selling eggs: $0\"], [\"Step 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\", \"Step 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\", \"Step 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\", \"Step 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars.\", \"The answer is: 18\\n\\n### Example 7\\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\\nStep 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\\nStep 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\\nStep 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars. The answer is: 18\\n\\n### Example 8\\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\\nStep 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\\nStep 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\\nStep 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars. The answer is: 18\\n\\n### Example 9\\nStep 1: Calculate the total number of eggs laid by the ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\\nStep 2: Calculate the number of eggs Janet eats and bakes: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\\nStep 3: Calculate the remaining eggs: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs\\nStep 4: Calculate the amount of money Janet makes from selling eggs: 9 * 2 = \u003c\u003c9*2=18\u003e\u003e18 dollars. The answer is: 18\"]]"
}
```

This subset can be loaded as:

```python
from datasets import load_dataset

ds = load_dataset("plaguss/test_math_shepherd_v3", "default")
```

Or simply as it follows, since there's only one configuration and is named `default`: 

```python
from datasets import load_dataset

ds = load_dataset("plaguss/test_math_shepherd_v3")
```


</details>





## References


```
@misc{wang2024mathshepherdverifyreinforcellms,
    title={Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations},
    author={Peiyi Wang and Lei Li and Zhihong Shao and R. X. Xu and Damai Dai and Yifei Li and Deli Chen and Y. Wu and Zhifang Sui},
    year={2024},
    eprint={2312.08935},
    archivePrefix={arXiv},
    primaryClass={cs.AI},
    url={https://arxiv.org/abs/2312.08935},
}
```


