task_categories:text2text-generation language:ar license:mit size_categories:1M<n<10M format:parquet modality:text library:datasets library:dask library:mlcroissant library:polars region:us Speech-Correction Diacritics Vocalized Arabic Tashkeel
---
dataset_info:
  features:
  - name: vocalized
    dtype: string
  - name: erroneous
    dtype: string
  splits:
  - name: train
    num_bytes: 4494447826
    num_examples: 1460625
  - name: valid
    num_bytes: 93024950
    num_examples: 30129
  - name: test
    num_bytes: 46293622
    num_examples: 15062
  download_size: 2248929902
  dataset_size: 4633766398
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: valid
    path: data/valid-*
  - split: test
    path: data/test-*
license: mit
task_categories:
- text2text-generation
language:
- ar
tags:
- Speech-Correction
- Diacritics
- Vocalized
- Arabic
- Tashkeel
---

# **Text with Diacritics Correction Dataset**

[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Dataset-yellow)](https://huggingface.co/datasets/FlouBsy/Text-with-Diacritics-Correction)

## **Overview**
This dataset is an **Arabic dataset** derived from [Abdou/arabic-tashkeel-dataset](https://huggingface.co/datasets/Abdou/arabic-tashkeel-dataset), with additional **preprocessing and error injection** to help train models for **speech and typing correction**.

### **Key Features**
✅ **Preprocessed Arabic text**: Cleaned `vocalized` column by removing non-Arabic characters, symbols, emojis, brackets, and numbers.  
✅ **Injected realistic errors**: Includes four types of errors that simulate real-world **speech and typing mistakes**.  
✅ **Large-scale dataset**: Over **1.5 million** text samples for robust training.  
✅ **Suitable for NLP tasks**: Useful for **text correction, speech recognition, and Arabic NLP research**.  

---

## **Dataset Details**

### **Statistics**
| Split | Samples |
|------------|---------:|
| Train | 1,460,625 |
| Validation | 30,129 |
| Test | 15,062 |
| **Total** | **1,505,816** |

### **Data Format**
- **File Type**: Parquet  
- **Columns**:  
  - `vocalized`: Preprocessed Arabic text  
  - `erroneous`: The same text with injected errors  

---

## **Error Injection Details**
We introduced **four types of common Arabic text errors** to enhance the dataset’s usability for speech and typing correction:

1. **Keyboard Errors** 🖥️  
   - Mistyped letters based on nearby keyboard keys.  
   - Example: `السلام` → `الستام` (minor keyboard slip).  

2. **Ordering Errors** 🔀  
   - Swapped adjacent letters.  
   - Example: `مرحبا` → `مرحاب` (common typing mistake).  

3. **Phonetic Errors** 🎙️  
   - Replaced letters with phonetically similar ones.  
   - Example: `ن` ↔ `م`, `س` ↔ `ث`.  

4. **Diacritic Errors** 🐪  
   - Altered or removed Arabic diacritics (`Tashkeel`).  
   - Example: `كَتَبَ` → `كَتُبَ`.  

These errors mimic **real-world mistakes** in **speech recognition and Arabic typing**, improving model robustness.

---

## **Usage**

### **Loading the Dataset in Python**
```python
from datasets import load_dataset

dataset = load_dataset("FlouBsy/Text-with-Diacritics-Correction")

# Example access
print(dataset['train'][0]) # Show first sample
```

## Collaborators

- **[Basma M.](https://huggingface.co/FlouBsy)** (Main Contributor)
- **[Selsabeel A.](https://huggingface.co/Selsabeel)**

## Citations

```
@misc{arabic_tashkeel_dataset,
  author = {Abdou},
  title = {{Arabic Tashkeel Dataset}},
  year = {2024},
  url = {https://huggingface.co/datasets/Abdou/arabic-tashkeel-dataset}
}

@misc{Text-with-Diacritics-Correction,
  author = {Basma M., Selsabeel A.},
  title = {{Arabic Speech & Typing Errors with Diacritics Dataset}},
  year = {2025},
  url = {https://huggingface.co/datasets/FlouBsy/Text-with-Diacritics-Correction}
}
```