size_categories:n<1K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: hash
    dtype: string
  - name: repo
    dtype: string
  - name: date
    dtype: string
  - name: license
    dtype: string
  - name: message
    dtype: string
  - name: mods
    list:
    - name: change_type
      dtype: string
    - name: old_path
      dtype: string
    - name: new_path
      dtype: string
    - name: diff
      dtype: string
  splits:
  - name: test
    num_bytes: 1515413
    num_examples: 163
  download_size: 532727
  dataset_size: 1515413
configs:
- config_name: default
  data_files:
  - split: test
    path: data/test-*
---
# BenchName (Commit message generation)

This is the benchmark for the Commit message generation task as part of the
[BenchName benchmark](https://huggingface.co/collections/icmlbenchname/icml-25-benchname-679b57e38a389f054cdf33e9).

All the repositories are published under permissive licenses (MIT, Apache-2.0, and BSD-3-Clause). The datapoints can be removed upon request.

## How-to

```py
from datasets import load_dataset

dataset = load_dataset("icmlbenchname/commit-message-generation", split="test")
```

Note that all the data we have is considered to be in the test split.

## About

#### Commits

Each example has the following fields:

| **Field** |              **Description**              |
|:---------:|:-----------------------------------------:|
|  `repo`   |            Commit repository.             |
|  `hash`   |               Commit hash.                |
|  `date`   |               Commit date.                |
| `license` |       Commit repository's license.        |
| `message` |              Commit message.              |
|  `mods`   | List of file modifications from a commit. |

Each file modification has the following fields:

|   **Field**   |                                          **Description**                                          |
|:-------------:|:-------------------------------------------------------------------------------------------------:|
| `change_type` | Type of change to current file. One of: `ADD`, `COPY`, `RENAME`, `DELETE`, `MODIFY` or `UNKNOWN`. |
|  `old_path`   |                           Path to file before change (might be empty).                            |
|  `new_path`   |                            Path to file after change (might be empty).                            |
|    `diff`     |                                   `git diff` for current file.                                    |

Data point example:

```json
{'hash': 'b76ed0db81b3123ede5dc5e5f1bddf36336f3722',
 'repo': 'apache/libcloud',
 'date': '05.03.2022 17:52:34',
 'license': 'Apache License 2.0',
 'message': 'Add tests which verify that all OpenStack driver can be instantiated\nwith all the supported auth versions.\nNOTE: Those tests will fail right now due to the regressions being\nintroduced recently which breaks auth for some versions.',
 'mods': [{'change_type': 'MODIFY',
    'new_path': 'libcloud/test/compute/test_openstack.py',
    'old_path': 'libcloud/test/compute/test_openstack.py',
    'diff': '@@ -39,6 +39,7 @@ from libcloud.utils.py3 import u\n<...>'}],
}    
```

### Dataset Structure

Each example has the following fields:

| **Field** |                          **Description**                           |
|:---------:|:------------------------------------------------------------------:|
|  `repo`   |                         Commit repository.                         |
|  `hash`   |                            Commit hash.                            |
|  `date`   |                            Commit date.                            |
| `license` |                    Commit repository's license.                    |
| `message` |                          Commit message.                           |
|  `label`  |         Label of the current commit as a target for CMG task.          |
| `comment` | Comment for a label for the current commit (optional, might be empty). |

Labels are in 1–5 scale, where:

* 1 – strong no
* 2 – weak no
* 3 – unsure
* 4 – weak yes
* 5 – strong yes

Data point example:

```json
{'hash': '1559a4c686ddc2947fc3606e1c4279062cc9480f',
 'repo': 'appscale/gts',
 'date': '15.07.2018 21:00:39',
 'license': 'Apache License 2.0',
 'message': 'Add auto_id_policy and logs_path flags\n\nThese changes were introduced in the 1.7.5 SDK.',
 'label': 1,
 'comment': 'no way to know the version'}
```

