size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars library:distilabel arxiv:2312.08935 region:us synthetic distilabel rlaif
---
size_categories: n<1K
dataset_info:
  features:
  - name: instruction
    dtype: string
  - name: answer
    dtype: string
  - name: golden_solution
    sequence: string
  - name: model_name
    dtype: string
  - name: solutions
    sequence: string
  - name: distilabel_metadata
    struct:
    - name: statistics_completer
      struct:
      - name: input_tokens
        sequence: int64
      - name: output_tokens
        sequence: int64
  - name: input
    dtype: string
  - name: label
    dtype: string
  splits:
  - name: train
    num_bytes: 38730
    num_examples: 15
  download_size: 25147
  dataset_size: 38730
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
tags:
- synthetic
- distilabel
- rlaif
---

<p align="left">
  <a href="https://github.com/argilla-io/distilabel">
    <img src="https://raw.githubusercontent.com/argilla-io/distilabel/main/docs/assets/distilabel-badge-light.png" alt="Built with Distilabel" width="200" height="32"/>
  </a>
</p>

# Dataset Card for test_math_shepherd_prm_temp05

This dataset has been created with [distilabel](https://distilabel.argilla.io/).


The pipeline script was uploaded to easily reproduce the dataset:
[pipe_math_shepherd.py](https://huggingface.co/datasets/plaguss/test_math_shepherd_prm_temp05/raw/main/pipe_math_shepherd.py).

It can be run directly using the CLI:

```console
distilabel pipeline run --script "https://huggingface.co/datasets/plaguss/test_math_shepherd_prm_temp05/raw/main/pipe_math_shepherd.py"
```


## Dataset Summary

This dataset contains a `pipeline.yaml` which can be used to reproduce the pipeline that generated it in distilabel using the `distilabel` CLI:

```console
distilabel pipeline run --config "https://huggingface.co/datasets/plaguss/test_math_shepherd_prm_temp05/raw/main/pipeline.yaml"
```

or explore the configuration:

```console
distilabel pipeline info --config "https://huggingface.co/datasets/plaguss/test_math_shepherd_prm_temp05/raw/main/pipeline.yaml"
```

## Dataset structure

The examples have the following structure per configuration:


<details><summary> Configuration: default </summary><hr>

```json
{
    "answer": "Janet sells 16 - 3 - 4 = \u003c\u003c16-3-4=9\u003e\u003e9 duck eggs a day.\nShe makes 9 * 2 = $\u003c\u003c9*2=18\u003e\u003e18 every day at the farmer\u2019s market.\n#### 18",
    "distilabel_metadata": {
        "statistics_completer": {
            "input_tokens": [
                146
            ],
            "output_tokens": [
                151
            ]
        }
    },
    "golden_solution": [
        "Step 1: Determine the number of eggs Janet has available for selling after subtracting the eggs she eats from the total eggs laid. She eats 3 eggs and the ducks lay 16 eggs per day, so 16 - 3 = \u003c\u003c16-3=13\u003e\u003e13 eggs are available for selling.",
        "Step 2: calculate the total revenue from selling the eggs. Each egg is sold for $2, so 13 * 2 = \u003c\u003c13*2=26\u003e\u003e26 dollars.",
        "The answer is: 26 dollars."
    ],
    "input": "Janet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers\u0027 market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers\u0027 market? Step 1: Calculate the total number of eggs laid by Janet\u0027s ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 egg\u043a\u0438\nStep 2: Calculate the total number of eggs eaten for breakfast and muffins: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 egg\u043a\u0438\nStep 3: Calculate the number of eggs left to sell: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs \u043a\u0438\nStep 4: Calculate the total amount earned from selling the eggs: 9 * $2 = \u003c\u003c9*2=18\u003e\u003e18 dollars The answer is: 18 \u043a\u0438",
    "instruction": "Janet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers\u0027 market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers\u0027 market?",
    "label": "Janet\u2019s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers\u0027 market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers\u0027 market? Step 1: Calculate the total number of eggs laid by Janet\u0027s ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs\nStep 2: Calculate the total number of eggs eaten for breakfast and muffins: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs\nStep 3: Calculate the number of eggs left to sell: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs -\nStep 4: Calculate the total amount earned from selling the eggs: 9 * $2 = \u003c\u003c9*2=18\u003e\u003e18 dollars The answer is: 18 -",
    "model_name": "meta-llama/Meta-Llama-3.1-8B-Instruct",
    "solutions": [
        "Step 1: Calculate the total number of eggs laid by Janet\u0027s ducks: 16 * 1 = \u003c\u003c16*1=16\u003e\u003e16 eggs",
        "Step 2: Calculate the total number of eggs eaten for breakfast and muffins: 3 + 4 = \u003c\u003c3+4=7\u003e\u003e7 eggs",
        "Step 3: Calculate the number of eggs left to sell: 16 - 7 = \u003c\u003c16-7=9\u003e\u003e9 eggs -",
        "Step 4: Calculate the total amount earned from selling the eggs: 9 * $2 = \u003c\u003c9*2=18\u003e\u003e18 dollars The answer is: 18 -"
    ]
}
```

This subset can be loaded as:

```python
from datasets import load_dataset

ds = load_dataset("plaguss/test_math_shepherd_prm_temp05", "default")
```

Or simply as it follows, since there's only one configuration and is named `default`: 

```python
from datasets import load_dataset

ds = load_dataset("plaguss/test_math_shepherd_prm_temp05")
```


</details>





## References


```
@misc{2312.08935,
Author = {Peiyi Wang and Lei Li and Zhihong Shao and R. X. Xu and Damai Dai and Yifei Li and Deli Chen and Y. Wu and Zhifang Sui},
Title = {Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations},
Year = {2023},
Eprint = {arXiv:2312.08935},
}
```


```
@misc{2312.08935,
Author = {Peiyi Wang and Lei Li and Zhihong Shao and R. X. Xu and Damai Dai and Yifei Li and Deli Chen and Y. Wu and Zhifang Sui},
Title = {Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations},
Year = {2023},
Eprint = {arXiv:2312.08935},
}
```


```
@misc{wang2024mathshepherdverifyreinforcellms,
    title={Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations},
    author={Peiyi Wang and Lei Li and Zhihong Shao and R. X. Xu and Damai Dai and Yifei Li and Deli Chen and Y. Wu and Zhifang Sui},
    year={2024},
    eprint={2312.08935},
    archivePrefix={arXiv},
    primaryClass={cs.AI},
    url={https://arxiv.org/abs/2312.08935},
}
```


```
@misc{wang2024mathshepherdverifyreinforcellms,
    title={Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations},
    author={Peiyi Wang and Lei Li and Zhihong Shao and R. X. Xu and Damai Dai and Yifei Li and Deli Chen and Y. Wu and Zhifang Sui},
    year={2024},
    eprint={2312.08935},
    archivePrefix={arXiv},
    primaryClass={cs.AI},
    url={https://arxiv.org/abs/2312.08935},
}
```


