language:fr size_categories:1K<n<10K format:parquet modality:tabular modality:text library:datasets library:pandas library:mlcroissant library:polars region:us
---
dataset_info:
  features:
  - name: File
    dtype: string
  - name: Date
    dtype: int64
  - name: Region_OCR
    dtype: string
  - name: Region_OCR_aligned
    dtype: string
  - name: Region_GT_aligned
    dtype: string
  - name: Sentence_OCR_aligned
    dtype: string
  - name: Sentence_GT_aligned
    dtype: string
  - name: Sentence_OCR
    dtype: string
  - name: Sentence_GT
    dtype: string
  - name: Distance
    dtype: int64
  - name: CER
    dtype: float64
  - name: WER
    dtype: float64
  - name: Sentence_OCR_corrupted
    dtype: string
  - name: corrupted_cer
    dtype: float64
  - name: corrupted_wer
    dtype: float64
  splits:
  - name: train
    num_bytes: 18654472
    num_examples: 2632
  - name: dev
    num_bytes: 2542628
    num_examples: 336
  - name: test
    num_bytes: 2031987
    num_examples: 301
  download_size: 6813801
  dataset_size: 23229087
configs:
- config_name: default
  data_files:
  - split: train
    path: data/train-*
  - split: dev
    path: data/dev-*
  - split: test
    path: data/test-*
language:
- fr
---

This dataset is a filtered version of the *ICDAR2017* Competition on Handwritten Text Recognition, focusing on monograph texts written between 1800 and 1900. It consists of a total of **957 documents**, divided into training, validation, and testing sets, and is designed for post-correction of OCR (Optical Character Recognition) text.

- **Training Set**: 2.63k phrases
- **Validation Set**: 336
- **Test Set**: 301

## Purpose
The dataset aims to improve the accuracy of digitized texts by providing a reliable Ground Truth for comparison and correction, specifically addressing the challenges of French text of 19th century.

## Dataset Columns Description

This dataset contains several columns with detailed information about OCR (Optical Character Recognition) outputs and their corresponding ground truths (GT). Below is a description of each column:

| Column Name               | Data Type    | Description                                                                                   |
|---------------------------|--------------|-----------------------------------------------------------------------------------------------|
| **File**                 | `string`     | Name or identifier of the file associated with the OCR and GT data.                          |
| **Date**                 | `int64`      | Date of the document, encoded as an integer.                                                 |
| **Region_OCR**           | `string`     | OCR-recognized region of text.                      |
| **Region_OCR_aligned**   | `string`     | Aligned OCR-recognized region for better comparison with GT data.                            |
| **Region_GT_aligned**    | `string`     | Aligned ground truth region for validation against OCR results.                              |
| **Sentence_OCR_aligned** | `string`     | Aligned OCR-recognized sentences for improved accuracy in comparisons.                       |
| **Sentence_GT_aligned**  | `string`     | Aligned ground truth sentences corresponding to OCR results.                                 |
| **Sentence_OCR**         | `string`     | Original OCR-recognized sentences, unaligned.                                                |
| **Sentence_GT**          | `string`     | Original ground truth sentences, unaligned.                                                 |
| **Distance**             | `int64`      | The computed edit distance between OCR sentences and ground truth.                           |
| **CER**                  | `float64`    | Character Error Rate (CER) for OCR sentences compared to the ground truth. Values range from 0 to 0.29. |
| **WER**                  | `float64`    | Word Error Rate (WER) for OCR sentences compared to the ground truth. Values range from 0 to 1.5. |
| **Sentence_OCR_corrupted** | `string`   | OCR sentences with additional corruption introduced for synthetic error analysis.            |
| **corrupted_cer**        | `float64`    | Character Error Rate for corrupted OCR sentences compared to the ground truth. Values range from 0 to 0.35. |
| **corrupted_wer**        | `float64`    | Word Error Rate for corrupted OCR sentences compared to the ground truth.                    |

This structured dataset is designed to analyze and evaluate OCR post-correction performance with both real and synthetic data.

## Author Information
Prepared by **Mikhail Biriuchinskii**, an engineer in Natural Language Processing at Sorbonne University.

## Original Dataset Reference
For more information, visit the original dataset source: [ICDAR2017 Competition on Post-OCR Text Correction](http://l3i.univ-larochelle.fr/ICDAR2017PostOCR).

## Copyright
The original corpus is publicly accessible, and I do not hold any rights to this deployment of the corpus.