language:en license:apache-2.0 size_categories:1K<n<10K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: apache-2.0
language:
- en
---

# Turtle Soup Dataset (English)

This dataset contains English Turtle Soup puzzles, designed for reverse thinking games.

## Dataset Overview

This dataset is an extension of [Duguce/TurtleBench1.5k](https://huggingface.co/datasets/Duguce/TurtleBench1.5k), aiming to provide high-quality reasoning data for **Turtle-soup Game**. The data covers various high-difficulty reasoning scenarios, supports both **Chinese** and **English**, and incorporates multiple augmentation methods to enhance diversity and logical consistency.

## Data Source

- The original dataset is from Hugging Face and is licensed under **Apache License 2.0**.
- The extended dataset is composed of translation-based, annotated, benchmark, and model-generated data, detailed below.

## Data Structure

The dataset includes the following fields, with each entry fully documenting the reasoning context and answer labels of a Turtle Soup story:

| Field Name   | Description                                                           |
|--------------|-----------------------------------------------------------------------|
| `id`         | A unique identifier for the story.                                    |
| `title`      | The title of the Turtle Soup story.                                   |
| `surface`    | The surface information of the story, i.e., clues directly available to the players. |
| `bottom`     | The deep background of the story, i.e., the context or answer that players need to infer. |
| `user_guess` | The player's assumption or guess about the story.                     |
| `label`      | Labels: `T` (True): correct guess, `F` (False): incorrect guess, `N` (Irrelevant): unrelated guess. |

## Data Distribution

### Original Data Label Distribution

The dataset contains **1500 entries**, distributed as follows:
- **T (True): 46.4%** - Indicates that the guess is correct.
- **F (False): 42.4%** - Indicates that the guess is incorrect.
- **N (Irrelevant): 11.2%** - Indicates that the guess is unrelated to the question.

### Augmentation Methods and Proportions

To enhance diversity and logical consistency, the following augmentation methods were applied:
1. **Translation-based Data: 59.0%**
   - Generated using translation techniques, such as translating English to Korean and back to English, ensuring semantic logic and content diversity.

2. **Manual Annotation: 20.0%**
   - Manually created data fulfilling the conditions of the N (Irrelevant) label to balance label distribution.

3. **Turtle Benchmark: 15.0%**
   - A benchmark specifically designed for Turtle-soup Game, covering various high-difficulty reasoning scenarios to provide baseline evaluation for inference abilities.

4. **Model Augmentation: 6.0%**
   - Initial data generated by models, refined through manual filtering to enhance diversity and coverage.

## Data Splits

The dataset is divided into training and testing sets, ensuring that stories in the testing set do not appear in the training set:
- **Training Set (Train Dataset):** 8000 entries
- **Testing Set (Test Dataset):** 2000 entries

## License

- The dataset is licensed under Apache License 2.0 and is intended for academic research only.
- If you use this dataset, please cite the original dataset: [Duguce/TurtleBench1.5k](https://huggingface.co/datasets/Duguce/TurtleBench1.5k).
