size_categories:1K<n<10K format:parquet modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2407.15694 region:us
---
dataset_info:
  features:
  - name: Headline
    dtype: string
  - name: article
    dtype: string
  - name: BARD responses
    dtype: string
  - name: GPT3.5_responses
    dtype: string
  - name: GPT4_responses
    dtype: string
  - name: ' Gemma_2B_Response'
    dtype: string
  - name: Gemma_7B_Response
    dtype: string
  splits:
  - name: BBC
    num_bytes: 56780976
    num_examples: 1762
  - name: NDTV
    num_bytes: 110217968
    num_examples: 5281
  download_size: 56391962
  dataset_size: 166998944
configs:
- config_name: default
  data_files:
  - split: BBC
    path: data/BBC-*
  - split: NDTV
    path: data/NDTV-*
---


# Counter Turing Test (CT²): Investigating AI-Generated Text Detection for Hindi
# AG_hi Dataset

## Paper Links
- [ArXiv Link](https://arxiv.org/abs/2407.15694)


---

## About the Dataset
The **AI-generated news article in Hindi (AG_hi) dataset** is introduced to assess the effectiveness of AI-generated text detection (AGTD) techniques for Hindi. 

---

## Dataset Overview
This dataset comprises two categories of Hindi news articles: human-written and AI-generated. The human-written articles were sourced from **BBC Hindi** and **NDTV** platforms, while the AI-generated articles were created using five selected LLMs: **GPT-4, GPT-3.5, BARD, Gemma-2B, and Gemma-7B**.

### Key Points:
- The dataset contains a total of **36,670** datapoints comprising of Human-written and AI-generated articles.
- **29,627 AI-generated articles** from **BBC** and **NDTV** were collected.

---

## Dataset Statistics

| Data Source | Human Written Articles | AI Generated Articles |
|-------------|------------------------|-----------------------|
| BBC         | 1,762                  | 7,390                 |
| NDTV        | 5,281                  | 22,237                |
| **Total**   | **7,043**              | **29,627**            |

| Model     | BBC  | NDTV  |
|-----------|------|-------|
| GPT-4     | 1762 | 5280  |
| GPT-3.5   | 1762 | 5280  |
| BARD      | 1762 | 5280  |
| Gemma-2B  | 468  | 1715  |
| Gemma-7B  | 1636 | 4679  |
| **Total** | 7390 | 22234 |


---

## Dataset Format and Structure
The dataset is divided into two parts, **BBC** and **NDTV**, each containing the following columns:

- **Headline**: The headline of the news article.
- **Article**: The full text of the human-written article.
- **BARD responses**
- **GPT3.5_responses**
- **GPT4_responses**
- **Gemma_2B_Response**
- **Gemma_7B_Response**

Each part of the dataset contains both human-written articles and AI-generated articles for direct comparison. This structure allows for detailed analysis of AGTD techniques.

---

## Usage
- The dataset is intended to be used for research purposes to further develop and evaluate AI-generated text detection techniques.
- It serves as a benchmark for multilingual AI text generation and detection, especially for regional languages like Hindi.

---

## Citation
If you use this dataset in your work, please cite:
```
@misc{kavathekar2024counterturingtestct2,
      title={Counter Turing Test ($CT^2$): Investigating AI-Generated Text Detection for Hindi -- Ranking LLMs based on Hindi AI Detectability Index ($ADI_{hi}$)}, 
      author={Ishan Kavathekar and Anku Rani and Ashmit Chamoli and Ponnurangam Kumaraguru and Amit Sheth and Amitava Das},
      year={2024},
      eprint={2407.15694},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2407.15694}, 
}
```