task_categories:text-classification task_categories:text-generation language:en license:cc-by-4.0 size_categories:100K<n<1M modality:text arxiv:2310.06825 arxiv:2403.08295 region:us
---
license: cc-by-4.0
task_categories:
- text-classification
- text-generation
language:
- en
pretty_name: Generated And Grounded Language Examples
size_categories:
- 100K<n<1M
---
# Dataset Card for Dataset Name

<!-- Provide a quick summary of the dataset. -->

Generated And Grounded Language Examples (GAGLE) consists of over 240,000 LLM-generated articles.
These articles differ by the model that generated them (Mistral-7B [1] or Gemma-2B [2]), the contextual information provided in the prompt, the decoding temperature, and the data domain (e.g.  science, news, etc).
Please refer to the paper below for more details.

## Dataset Details

The seeds for all articles are sourced from five academic datasets:


1. `wikipedia` [3],

2. `big_patent` [4], consisting of over one million records of U.S. patent documents,

3. `newsroom` [5], containing over one million news articles,

4. `scientific_papers` [6], a collection of research papers obtained from ArXiv and PubMed OpenAccess repositories,

5. `billsum` [7], containing US Congressional and California state bills.

So, all articles are of encyclopedic, news, legal, scientific or patents nature. The dataset contains, at most, 1,000 articles from each domain.


The articles are generated via the following prompting strategies:

1. `continue_pt`: Simple continuation based on a short prefix (no prompting) using pretrained model.

2. `continue_it`: Simple continuation based on a short prefix (no prompting) using instruction-tuned model.
  
3. `cot`: Ask the model to generate an summary first before generating the article. A short prefix is provided.
  
4. `short_keywords`: A few, unordered list of keywords.
  
5. `keywords`: Many unordered keywords.
  
6. `summary`: A summary of the entire article.
  
7. `summary_keywords`: Both a summary and an unordered list of many keywords.
  
8. `excerpt`: An ordered list of long excerpts from the original article.


With the exception of `continue_pt`, all other prompts are used in instruction-tuned models. 

### Dataset Description

<!-- Provide a longer summary of what this dataset is. -->

- **Language(s) (NLP):** English
- **License:** CC-by-4.0


## Intended Uses

<!-- Address questions around how the dataset is intended to be used. -->

Research in domains related to detecting and analyzing LLM-generated tests.


## Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->

A single data record has the following structure:
```
{
  '_id': wikip-gemma-2b-extend-0.5-00013-of-00256--513,
  'dataset_name': wikipedia,
  'dataset_id': 00013-of-00256__513,
  'model': gemma_2b,
  'prompt_type': continue_it,
  'temperature': 0.5,
  'prefix': '(the following article congtains a minimum of 15 p ... began boxing at the age of 14 and also competed in',
  'llm_text': ' wrestling. He was a two-time state champion in wr ... del, and an inspiration to all who love the sport.',
  'original_text': 'Anthony Morrison (born March 29, 1984) is an Ameri ... vania\nCategory:Featherweight mixed martial artists',
  'llm_text_quality': 'The document provides a detailed timeline of Antho ... efit from additional analysis and depth. Rating: 3',
  'scores': {
    'gemma-2b': {
      'llm_text': {
        'token_ids': [50244, 235265,  ...  , 235265],
        'token_texts': [' wrestling', '.',  ...  , '.'],
        'token_scores': [-5.854015350341797, -8.04088020324707,  ...  , -0.056363798677921295],
      }
      'original_text': {
        'token_ids': [44270, 46035,  ...  , 14880],
        'token_texts': ['Anthony', ' Morrison',  ...  , ' artists'],
        'token_scores': [-9.958677291870117, -7.5479841232299805,  ...  , -0.012911745347082615],
      }
    }
    'mistral-7b': {
      'llm_text': {
        'token_ids': [28705, 17619,  ...  , 28723],
        'token_texts': ['', 'wrest',  ...  , '.'],
        'token_scores': [-6.125, -7.25,  ...  , -0.0155029296875],
      }
      'original_text': {
        'token_ids': [15089, 4151,  ...  , 10611],
        'token_texts': ['Anthony', 'Mor',  ...  , 'artists'],
        'token_scores': [-10.0, -6.1875,  ...  , -0.0155029296875],
      }
    }
  }
}
```


## Citation

If you use this dataset, please cite the following paper: 

**BibTeX:**

```
@misc{alabdulmohsin_tale_two_fractal_structures2025,
      title={A Tale of Two Structures: Do LLMs  Capture the Fractal Complexity of Language?}, 
      author={Ibrahim Alabdulmohsin and Andreas Steiner},
      year={2025},
}
```


## References

[1] Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., ... & Sayed, W. E. (2023). Mistral 7B. arXiv preprint arXiv:2310.06825.

[2] Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., ... & Kenealy, K. (2024). Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295.

[3] Wikimedia, Downloads. (2024).  URL: https://dumps.wikimedia.org.

[4] Sharma, E., Li, C., and Wang, L. (2019). BIGPATENT: A large-scale dataset for abstractive and coherent summarization.

[5] Grusky,  M.,  Naaman,  M.,  and  Artzi,  Y. (2018). Newsroom: A dataset  of  1.3  million  summaries  with  diverse  extractive  strategies. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics.

[6] Cohan,  A.,  Dernoncourt,  F.,  Kim,  D.  S.,  Bui,  T.,  Kim,S.,  Chang,  W.,  and  Goharian,  N. (2018). A  discourse-aware attention model for abstractive summarization of long documents. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics.

[7] Kornilova,  A. and Eidelman,  V. (2019).  Billsum:  A corpus for automatic summarization of US legislation.

## Dataset Card Contact

Ibrahim Alabdulmohsin: ibomohsin@google.com
