task_categories:text-classification language:en license:unknown size_categories:100K<n<1M format:csv modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
license: unknown
task_categories:
- text-classification
language:
- en
size_categories:
- 100K<n<1M
---
## Dataset Creation

The AG-Pair dataset is built from the original dataset AG's News that contains 120k training samples from four topics.
Given a pair of news as input, the model has to predict whether they are belonging to the same topic (Same) or not (Different).

To generate this dataset, samples in AG are iterated in random order and have an equal chance to combine a sample in the same topic or the other three topics.
Thus the numbers of training samples in two classes are both 60k.
Moreover, each news in AG's News occurs exactly twice in the AG-Pair dataset to keep the same word frequency.

## Additional Information

### Dataset Curators

[Chong Li](lichong2021@ia.ac.cn)


### Citation Information


```
@inproceedings{li-etal-2023-FunctionalSpecialization,
  author    = {Chong Li and
               Shaonan Wang and
               Yunhao Zhang and
               Jiajun Zhang and
               Chengqing Zong},
  title = "Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning",
  booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
  year = "2023",
  address = "Singapore",
  publisher = "Association for Computational Linguistics",
}
```
