size_categories:n<1K format:parquet modality:tabular modality:text library:datasets library:pandas library:mlcroissant library:polars library:distilabel arxiv:2310.01377 region:us synthetic distilabel rlaif
---
size_categories: n<1K
dataset_info:
  features:
  - name: instruction
    dtype: string
  - name: topic
    dtype: string
  - name: generations
    sequence: string
  - name: distilabel_metadata
    struct:
    - name: raw_output_ultra_feedback_0
      dtype: string
  - name: model_names
    sequence: string
  - name: ratings
    sequence: int64
  - name: rationales
    sequence: string
  - name: model_name
    dtype: string
  - name: prompt
    dtype: string
  - name: prompt_id
    dtype: string
  - name: chosen
    list:
    - name: content
      dtype: string
    - name: role
      dtype: string
  - name: chosen_rating
    dtype: int64
  - name: rejected
    list:
    - name: content
      dtype: string
    - name: role
      dtype: string
  - name: rejected_rating
    dtype: int64
  splits:
  - name: train
    num_bytes: 250512
    num_examples: 20
  download_size: 204921
  dataset_size: 250512
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
tags:
- synthetic
- distilabel
- rlaif
---

<p align="left">
  <a href="https://github.com/argilla-io/distilabel">
    <img src="https://raw.githubusercontent.com/argilla-io/distilabel/main/docs/assets/distilabel-badge-light.png" alt="Built with Distilabel" width="200" height="32"/>
  </a>
</p>

# Dataset Card for example-preference-dataset

This dataset has been created with [distilabel](https://distilabel.argilla.io/).



## Dataset Summary

This dataset contains a `pipeline.yaml` which can be used to reproduce the pipeline that generated it in distilabel using the `distilabel` CLI:

```console
distilabel pipeline run --config "https://huggingface.co/datasets/Yuriy81/example-preference-dataset/raw/main/pipeline.yaml"
```

or explore the configuration:

```console
distilabel pipeline info --config "https://huggingface.co/datasets/Yuriy81/example-preference-dataset/raw/main/pipeline.yaml"
```

## Dataset structure

The examples have the following structure per configuration:


<details><summary> Configuration: default </summary><hr>

```json
{
    "chosen": [
        {
            "content": "How can I create an efficient and robust workflow that utilizes advanced automation techniques to extract targeted data, including customer information, from diverse PDF documents and effortlessly integrate it into a designated Google Sheet? Furthermore, I am interested in establishing a comprehensive and seamless system that promptly activates an SMS notification on my mobile device whenever a new PDF document is uploaded to the Google Sheet, ensuring real-time updates and enhanced accessibility.",
            "role": "user"
        },
        {
            "content": "Creating an efficient and robust workflow that automates the extraction of targeted data from PDFs and integrates it into a Google Sheet, while also setting up SMS notifications for new uploads, involves several steps. Here\u0027s a comprehensive guide to achieve this:\n\n### Step 1: Extract Data from PDFs\n\n1. **Select a PDF Extraction Tool**:\n   - Use tools like Adobe Acrobat, Tabula, or Python libraries such as `PyPDF2`, `pdfminer.six`, or `pytesseract` for OCR (Optical Character Recognition).\n\n2. **Automate Extraction**:\n   - If you are comfortable with coding, you can use Python to automate the extraction process. Here\u0027s a basic example using `PyPDF2`:\n\n   ```python\n   import PyPDF2\n\n   def extract_text_from_pdf(pdf_path):\n       with open(pdf_path, \u0027rb\u0027) as file:\n           reader = PyPDF2.PdfFileReader(file)\n           text = \u0027\u0027\n           for page_num in range(reader.numPages):\n               page = reader.getPage(page_num)\n               text += page.extract_text()\n       return text\n   ```\n\n### Step 2: Parse and Structure Data\n\n1. **Define Parsing Logic**:\n   - Use regular expressions or specific parsing libraries to extract targeted data like customer information.\n\n   ```python\n   import re\n\n   def parse_customer_info(text):\n       # Assuming customer info follows a certain pattern\n       customer_info = re.findall(r\u0027Customer Name: (.*)\\nAddress: (.*)\\nEmail: (.*)\\n\u0027, text)\n       return customer_info\n   ```\n\n### Step 3: Integrate Data into Google Sheets\n\n1. **Use Google Sheets API**:\n   - Set up Google Sheets API and use a service account to authenticate.\n\n   ```python\n   from googleapiclient.discovery import build\n   from google.oauth2.service_account import Credentials\n\n   SCOPES = [\u0027https://www.googleapis.com/auth/spreadsheets\u0027]\n   SERVICE_ACCOUNT_FILE = \u0027path/to/credentials.json\u0027\n\n   creds = Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)\n\n   service = build(\u0027sheets\u0027, \u0027v4\u0027, credentials=creds)\n   sheet = service.spreadsheets()\n\n   SPREADSHEET_ID = \u0027your_spreadsheet_id\u0027\n   RANGE_NAME = \u0027Sheet1!A1\u0027\n\n   def update_google_sheet(data):\n       body = {\n           \u0027values\u0027: data\n       }\n       result = sheet.values().update(spreadsheetId=",
            "role": "assistant"
        }
    ],
    "chosen_rating": 4,
    "distilabel_metadata": {
        "raw_output_ultra_feedback_0": "#### Output for Text 1\nRating: 4\nRationale: Text 1 provides a detailed and structured guide to creating an automated workflow for extracting data from PDFs and integrating it into Google Sheets, closely aligning with the instruction\u0027s requirements. It includes technical specifics like tool recommendations and code snippets which enhance its informativeness and correctness. However, the text is truncated and does not complete the SMS notification setup, which is crucial for fulfilling the user\u0027s intent fully. Despite this, the provided content is accurate and truthful, with no evidence of hallucination. The minor lack of completion in addressing the SMS notifications slightly affects its score.\n\n#### Output for Text 2\nRating: 4\nRationale: Text 2 offers a thorough guide consistent with the instruction\u0027s aim, detailing the process of data extraction, integration into Google Sheets, and the setup for SMS notifications. It mentions advanced automation techniques and provides relevant examples, which is highly informative. The text uses appropriate tools and technologies, such as Python libraries and Google Sheets API, ensuring correctness. However, like Text 1, it is also truncated and does not complete the explanation for setting up SMS notifications. This partial information slightly reduces its effectiveness in fully meeting the user\u0027s needs, although what is provided is accurate and detailed."
    },
    "generations": [
        "Creating an efficient and robust workflow that automates the extraction of targeted data from PDFs and integrates it into a Google Sheet, while also setting up SMS notifications for new uploads, involves several steps. Here\u0027s a comprehensive guide to achieve this:\n\n### Step 1: Extract Data from PDFs\n\n1. **Select a PDF Extraction Tool**:\n   - Use tools like Adobe Acrobat, Tabula, or Python libraries such as `PyPDF2`, `pdfminer.six`, or `pytesseract` for OCR (Optical Character Recognition).\n\n2. **Automate Extraction**:\n   - If you are comfortable with coding, you can use Python to automate the extraction process. Here\u0027s a basic example using `PyPDF2`:\n\n   ```python\n   import PyPDF2\n\n   def extract_text_from_pdf(pdf_path):\n       with open(pdf_path, \u0027rb\u0027) as file:\n           reader = PyPDF2.PdfFileReader(file)\n           text = \u0027\u0027\n           for page_num in range(reader.numPages):\n               page = reader.getPage(page_num)\n               text += page.extract_text()\n       return text\n   ```\n\n### Step 2: Parse and Structure Data\n\n1. **Define Parsing Logic**:\n   - Use regular expressions or specific parsing libraries to extract targeted data like customer information.\n\n   ```python\n   import re\n\n   def parse_customer_info(text):\n       # Assuming customer info follows a certain pattern\n       customer_info = re.findall(r\u0027Customer Name: (.*)\\nAddress: (.*)\\nEmail: (.*)\\n\u0027, text)\n       return customer_info\n   ```\n\n### Step 3: Integrate Data into Google Sheets\n\n1. **Use Google Sheets API**:\n   - Set up Google Sheets API and use a service account to authenticate.\n\n   ```python\n   from googleapiclient.discovery import build\n   from google.oauth2.service_account import Credentials\n\n   SCOPES = [\u0027https://www.googleapis.com/auth/spreadsheets\u0027]\n   SERVICE_ACCOUNT_FILE = \u0027path/to/credentials.json\u0027\n\n   creds = Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)\n\n   service = build(\u0027sheets\u0027, \u0027v4\u0027, credentials=creds)\n   sheet = service.spreadsheets()\n\n   SPREADSHEET_ID = \u0027your_spreadsheet_id\u0027\n   RANGE_NAME = \u0027Sheet1!A1\u0027\n\n   def update_google_sheet(data):\n       body = {\n           \u0027values\u0027: data\n       }\n       result = sheet.values().update(spreadsheetId=",
        "Creating an efficient and robust workflow for extracting targeted data from PDF documents, integrating that data into Google Sheets, and setting up SMS notifications involves several steps. Below is a comprehensive guide that outlines how you can achieve this using advanced automation techniques:\n\n### Step 1: Extracting Data from PDF Documents\n\n1. **Choose a PDF Extraction Tool:**\n   - Use tools like **Adobe Acrobat**, **Tabula**, or **PDF.co** for simple extraction.\n   - For more advanced automation, consider using **Python libraries** such as `PyPDF2`, `pdfplumber`, or `PyMuPDF` for custom extraction scripts.\n\n2. **Set Up a PDF Processing Script:**\n   - Write a Python script that extracts customer information from PDFs.\n   - Use regular expressions to target specific data formats (e.g., names, emails, phone numbers).\n\n3. **Automate Extraction:**\n   - Schedule the script to run periodically using a task scheduler (e.g., cron jobs for Linux or Task Scheduler for Windows).\n   - Alternatively, consider using **Google Cloud Functions** or **AWS Lambda** for serverless execution triggered by new uploads.\n\n### Step 2: Integrating Data into Google Sheets\n\n1. **Set Up Google Sheets API:**\n   - Enable the Google Sheets API in the Google Cloud Console.\n   - Create credentials (OAuth 2.0) and download the `credentials.json` file.\n\n2. **Write Integration Logic:**\n   - Extend your Python script to include the Google Sheets API:\n     - Use the `gspread` library to connect and write data to your Google Sheet.\n     - Format the extracted data as needed and append it to the designated sheet.\n   \n   Example snippet for updating Google Sheets:\n   ```python\n   import gspread\n   from oauth2client.service_account import ServiceAccountCredentials\n\n   # Define the scope and authorize\n   scope = [\"https://spreadsheets.google.com/feeds\", \"https://www.googleapis.com/auth/drive\"]\n   creds = ServiceAccountCredentials.from_json_keyfile_name(\u0027credentials.json\u0027, scope)\n   client = gspread.authorize(creds)\n\n   # Open the Google Sheet and append data\n   sheet = client.open(\"Your Google Sheet Name\").sheet1\n   sheet.append_row([\u0027Extracted Data 1\u0027, \u0027Extracted Data 2\u0027, \u0027Extracted Data 3\u0027])\n   ```\n\n### Step 3: Setting Up SMS Notifications\n\n1. **Choose an SMS Service:**\n   - Use services like **Twilio**, **"
    ],
    "instruction": "How can I create an efficient and robust workflow that utilizes advanced automation techniques to extract targeted data, including customer information, from diverse PDF documents and effortlessly integrate it into a designated Google Sheet? Furthermore, I am interested in establishing a comprehensive and seamless system that promptly activates an SMS notification on my mobile device whenever a new PDF document is uploaded to the Google Sheet, ensuring real-time updates and enhanced accessibility.",
    "model_name": "gpt-4-turbo",
    "model_names": [
        "gpt-4o",
        "gpt-4o-mini"
    ],
    "prompt": "How can I create an efficient and robust workflow that utilizes advanced automation techniques to extract targeted data, including customer information, from diverse PDF documents and effortlessly integrate it into a designated Google Sheet? Furthermore, I am interested in establishing a comprehensive and seamless system that promptly activates an SMS notification on my mobile device whenever a new PDF document is uploaded to the Google Sheet, ensuring real-time updates and enhanced accessibility.",
    "prompt_id": "1fe12b572e62e24092e3256cdc16d6adc036ea88ed6b3af227d7fa2f54dbd0c5",
    "ratings": [
        4,
        4
    ],
    "rationales": [
        "Text 1 provides a detailed and structured guide to creating an automated workflow for extracting data from PDFs and integrating it into Google Sheets, closely aligning with the instruction\u0027s requirements. It includes technical specifics like tool recommendations and code snippets which enhance its informativeness and correctness. However, the text is truncated and does not complete the SMS notification setup, which is crucial for fulfilling the user\u0027s intent fully. Despite this, the provided content is accurate and truthful, with no evidence of hallucination. The minor lack of completion in addressing the SMS notifications slightly affects its score.",
        "Text 2 offers a thorough guide consistent with the instruction\u0027s aim, detailing the process of data extraction, integration into Google Sheets, and the setup for SMS notifications. It mentions advanced automation techniques and provides relevant examples, which is highly informative. The text uses appropriate tools and technologies, such as Python libraries and Google Sheets API, ensuring correctness. However, like Text 1, it is also truncated and does not complete the explanation for setting up SMS notifications. This partial information slightly reduces its effectiveness in fully meeting the user\u0027s needs, although what is provided is accurate and detailed."
    ],
    "rejected": [
        {
            "content": "How can I create an efficient and robust workflow that utilizes advanced automation techniques to extract targeted data, including customer information, from diverse PDF documents and effortlessly integrate it into a designated Google Sheet? Furthermore, I am interested in establishing a comprehensive and seamless system that promptly activates an SMS notification on my mobile device whenever a new PDF document is uploaded to the Google Sheet, ensuring real-time updates and enhanced accessibility.",
            "role": "user"
        },
        {
            "content": "Creating an efficient and robust workflow that automates the extraction of targeted data from PDFs and integrates it into a Google Sheet, while also setting up SMS notifications for new uploads, involves several steps. Here\u0027s a comprehensive guide to achieve this:\n\n### Step 1: Extract Data from PDFs\n\n1. **Select a PDF Extraction Tool**:\n   - Use tools like Adobe Acrobat, Tabula, or Python libraries such as `PyPDF2`, `pdfminer.six`, or `pytesseract` for OCR (Optical Character Recognition).\n\n2. **Automate Extraction**:\n   - If you are comfortable with coding, you can use Python to automate the extraction process. Here\u0027s a basic example using `PyPDF2`:\n\n   ```python\n   import PyPDF2\n\n   def extract_text_from_pdf(pdf_path):\n       with open(pdf_path, \u0027rb\u0027) as file:\n           reader = PyPDF2.PdfFileReader(file)\n           text = \u0027\u0027\n           for page_num in range(reader.numPages):\n               page = reader.getPage(page_num)\n               text += page.extract_text()\n       return text\n   ```\n\n### Step 2: Parse and Structure Data\n\n1. **Define Parsing Logic**:\n   - Use regular expressions or specific parsing libraries to extract targeted data like customer information.\n\n   ```python\n   import re\n\n   def parse_customer_info(text):\n       # Assuming customer info follows a certain pattern\n       customer_info = re.findall(r\u0027Customer Name: (.*)\\nAddress: (.*)\\nEmail: (.*)\\n\u0027, text)\n       return customer_info\n   ```\n\n### Step 3: Integrate Data into Google Sheets\n\n1. **Use Google Sheets API**:\n   - Set up Google Sheets API and use a service account to authenticate.\n\n   ```python\n   from googleapiclient.discovery import build\n   from google.oauth2.service_account import Credentials\n\n   SCOPES = [\u0027https://www.googleapis.com/auth/spreadsheets\u0027]\n   SERVICE_ACCOUNT_FILE = \u0027path/to/credentials.json\u0027\n\n   creds = Credentials.from_service_account_file(SERVICE_ACCOUNT_FILE, scopes=SCOPES)\n\n   service = build(\u0027sheets\u0027, \u0027v4\u0027, credentials=creds)\n   sheet = service.spreadsheets()\n\n   SPREADSHEET_ID = \u0027your_spreadsheet_id\u0027\n   RANGE_NAME = \u0027Sheet1!A1\u0027\n\n   def update_google_sheet(data):\n       body = {\n           \u0027values\u0027: data\n       }\n       result = sheet.values().update(spreadsheetId=",
            "role": "assistant"
        }
    ],
    "rejected_rating": 4,
    "topic": "Software Development"
}
```

This subset can be loaded as:

```python
from datasets import load_dataset

ds = load_dataset("Yuriy81/example-preference-dataset", "default")
```

Or simply as it follows, since there's only one configuration and is named `default`: 

```python
from datasets import load_dataset

ds = load_dataset("Yuriy81/example-preference-dataset")
```


</details>



## References


```
@misc{cui2024ultrafeedbackboostinglanguagemodels,
    title={UltraFeedback: Boosting Language Models with Scaled AI Feedback},
    author={Ganqu Cui and Lifan Yuan and Ning Ding and Guanming Yao and Bingxiang He and Wei Zhu and Yuan Ni and Guotong Xie and Ruobing Xie and Yankai Lin and Zhiyuan Liu and Maosong Sun},
    year={2024},
    eprint={2310.01377},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2310.01377},
}
```


