language:en language:fa language:zh language:ja language:hi language:id language:ha language:ur license:cc-by-nc-4.0 size_categories:1K<n<10K format:csv modality:text library:datasets library:pandas library:mlcroissant library:polars arxiv:2502.14315 region:us
---
license: cc-by-nc-4.0
language:
- en
- fa
- zh
- ja
- hi
- id
- ha
- ur
configs:
  - config_name: Procedures
    data_files:
      - split: Procedures
        path: Procedures.csv
  - config_name: MCQs
    data_files:
      - split: MCQs
        path: MCQs.csv
  - config_name: Conversations
    data_files:
      - split: Conversations
        path: Conversations.csv
---

<h2 align="left">
    CAPTex: A Benchmark for Culturally-Aware Procedural Text Understanding
</h2>
<p align="left"> <i>Amir Hossein Yari, Fajri Koto </i></p>
<h4 align="left">
    Sharif University of Technology, MBZUAI
</h4>

---

# Introduction

CAPTex (Culturally-Aware Procedural Texts) is a dataset designed to evaluate the ability of multilingual large language models (mLLMs) to comprehend and reason about procedural texts embedded in diverse cultural contexts. The dataset includes procedural knowledge from seven culturally distinct regions: China, India, Indonesia, Iran, Japan, Nigeria, and Pakistan. To ensure authenticity and cultural richness, native speakers from each region contributed to the dataset, providing both original procedural texts and their English translations.

---

# Statistics 📊

CAPTex consists of 1,400 human-written procedural texts (100 texts per language, spanning 7 native languages, plus their English translations), 700 structured conversations based on procedural texts, and 5600 multiple-choice questions (MCQs). The dataset covers 10 culturally significant domains:

1. Food and cuisine
2. Celebrations and festivals
3. Social etiquette and hospitality
4. Craftsmanship and artisan skills
5. Traditional attire and dress
6. Agricultural and seasonal practices
7. Religious and spiritual practices
8. Life milestones and family rites
9. Sports, games, and competitions
10. Environmental and nature-based practices

### Language Distribution 🌍

CAPTex includes procedural texts in the following languages, categorized by resource availability:

- **High-resource languages:** Chinese (Mandarin), Japanese, English
- **Mid-resource languages:** Persian, Hindi
- **Low-resource languages:** Indonesian, Urdu, Hausa

# Licence 📜

As a work constituting a collection of documents that have been cleaned, structured, annotated, and otherwise processed, the CAPTex dataset is licensed under the Creative Commons Attribution 4.0 International License. This license permits use, sharing, adaptation, distribution, and reproduction in any medium or format, provided that appropriate credit is given to the original author and source, a link to the Creative Commons license is provided, and any modifications made are indicated.

For more details, visit [Creative Commons License](https://creativecommons.org/licenses/by/4.0/).

# Citation 📝

Please find out paper <a href="https://arxiv.org/abs/2502.14315" target="_blank" style="margin-right: 15px; margin-left: 10px">here.</a>
```
@misc{yari2025unveilingculturalblindspots,
      title={Unveiling Cultural Blind Spots: Analyzing the Limitations of mLLMs in Procedural Text Comprehension}, 
      author={Amir Hossein Yari and Fajri Koto},
      year={2025},
      eprint={2502.14315},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.14315}, 
}
```