size_categories:1K<n<10K format:parquet modality:tabular modality:text library:datasets library:dask library:mlcroissant library:polars region:us trl-logs grpo-completions
---
tags:
- trl-logs
- grpo-completions
---

# TRL GRPO Completion logs

This dataset contains the completions generated during training using `trl` and GRPO.

The completions are stored in parquet files, and each file contains the completions for a single step of training (depending on the `logging_steps` argument).

Each file contains the following columns:
- `step`: the step of training
- `prompt`: the prompt used to generate the completion
- `completion`: the completion generated by the model
- `reward`: the reward given to the completion by all the reward functions

Additionally, the individual reward scores are also logged using the column naming convention `reward_<reward_function_name>`. This can be used to better understand how the reward functions are working and how they interact with each other. 

Having this data stored as a simple parquet file makes it easy to load and analyze using the Datasets Viewer, Polars, Pandas, etc.

You can load the dataset using the `datasets` library:

```python
import datasets

dataset = datasets.load_dataset(davanstrien/test-logs)
```

You can also load the dataset using Polars:

```python
import polars as pl

# Login using e.g. `huggingface-cli login` to access this dataset if it's private
df = pl.read_parquet(f'hf://datasets/davanstrien/test-logs/**/*.parquet')
```

