{'fraction_non_alphanumeric': 0.02320185614849188, 'fraction_numerical': 0.0, 'mean_word_length': 5.488721804511278, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Extreme value theory based text binarization in documents and natural scenes

This paper presents a novel image binarization method that can deal with degradations such as shadows, non-uniform illumination, low-contrast, large signal-dependent noise, smear and strain. A pre-processing procedure based on morphological operations is first applied to suppress light/dark structures connected to image border. A novel binarization concept based on difference of gamma functions is presented. Next Generalized Extreme Value Distribution (GEVD) is used to find proper threshold for binarization with a significance level. Proposed method emphasizes on region of interest (with the help of morphological operations) and generates less noisy artifacts (due to GEVD). It is much simpler than other methods and works better on degraded documents and natural scene images