size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: idx
    dtype: int64
  - name: response_source
    dtype: string
  - name: category
    dtype: string
  - name: instruction
    dtype: string
  - name: response
    dtype: string
  splits:
  - name: train
    num_bytes: 970116
    num_examples: 320
  download_size: 398377
  dataset_size: 970116
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
---

## Description

This dataset is used to check criticon prompts/responses while testing, it contains instructions/responses from mt_bench_eval, as extracted from: https://github.com/kaistAI/prometheus/blob/main/evaluation/benchmark/data/mt_bench_eval.json

The dataset has been obtained cleaning the data with:

```python
import re
import pandas as pd
from datasets import Dataset

df = pd.read_json("mt_bench_eval.json", lines=True)

ds = Dataset.from_pandas(df, preserve_index=False)


def prepare_ds(example):
    example["instruction"].split("###The instruction to evaluate:")
    pat1 = r'###The instruction to evaluate:(.*?)###Response to evaluate:'
    pat2 = r'###Response to evaluate:(.*?)###Reference Answer'
    matches_1 = re.search(pat1, example["instruction"], re.DOTALL)
    matches_2 = re.search(pat2, example["instruction"], re.DOTALL)
    instruction = response = None
    if matches_1:
        instruction = matches_1.group(1).strip()
    if matches_2:
        response = matches_2.group(1).strip()
    return {
        "instruction": instruction,
        "response": response
    }

ds = ds.map(prepare_ds)
```
