task_categories:question-answering task_categories:text-generation language:zh license:apache-2.0 size_categories:1K<n<10K format:json modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: apache-2.0
task_categories:
- question-answering
- text-generation
language:
- zh
size_categories:
- 1K<n<10K
---

# DeepSeek-R1 Ruozhiba Dataset

This repository contains responses generated using Together AI's DeepSeek-R1 API for the [Ruozhiba GPT4 dataset](https://huggingface.co/datasets/hfl/ruozhiba_gpt4). The dataset consists of 4,898 question-answer pairs in Chinese.

## Dataset Details

- **Source**: Original questions from [hfl/ruozhiba_gpt4](https://huggingface.co/datasets/hfl/ruozhiba_gpt4)
- **Model**: DeepSeek-R1 via Together AI API
- **System Prompt**: Adapted from [bespokelabs/Bespoke-Stratos-17k](https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k?row=0)
- **Format**: JSON
- **Size**: 4,898 entries

## Known Limitations

During the response generation process using DeepSeek-R1, we observed several limitations:

1. **Mixed Language Output**: Despite the questions being in Chinese, responses sometimes contain a mix of Chinese and English content, even when not prompted to do so.

2. **Inconsistent Thought Process**: Although the system prompt explicitly requires the model to follow a structured thinking process with `<|begin_of_thought|>` and `<|end_of_thought|>` markers, the model sometimes:
   - Skips the thinking process entirely
   - Provides minimal or superficial thinking steps
   - Jumps directly to solutions without showing work

3. **Prompt Adherence**: While using the Bespoke dataset's system prompt format, which emphasizes systematic thinking and analysis, the model doesn't consistently maintain this structured approach across all responses.

## Data Format

Each entry in the dataset follows this structure:

```json

{
"system": "...[system prompt]...",
"conversation": [
{
"from": "user",
"value": "[question in Chinese]"
},
{
"from": "assistant",
"value": "[DeepSeek-R1 response]"
}
]
}
```


## Usage Notes

This dataset can be useful for:
- Doing the SFT to make the R1-like model. 
- Studying Chinese language generation capabilities
- Evaluating structured thinking in AI responses
- Comparing different models' performance on the same prompts

## Citation

If you use this dataset, please cite the original sources: 

```
bibtex
@misc{hfl2024ruozhiba,
title={Ruozhiba GPT4 Dataset},
author={HFL},
year={2024},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/datasets/hfl/ruozhiba_gpt4}}
}
@misc{bespokelabs2024stratos,
title={Bespoke Stratos 17k Dataset},
author={Bespoke Labs},
year={2024},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/datasets/bespokelabs/Bespoke-Stratos-17k}}
}
```