{'fraction_non_alphanumeric': 0.012121212121212121, 'fraction_numerical': 0.0, 'mean_word_length': 5.6066666666666665, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using connected component decomposition to detect straight line segments in documents

Straight line segment detection in digital documents has been studied extensively for the past few decades. One of the challenges is to detect line segments without priori information about document images and render good results without much parameter calibration. In this paper, we introduce a novel algorithm that is simple but effective in detecting straight line segments in scanned documents. Our Connected Component Decomposition (CCD) approach first decomposes the connected components based on the gradient direction of the edge contours, and then uses Chebyshev's inequality to statistically distinguish lines from characters, followed by a simple post processing step to examine straightness of remain segments. This CCD approach is simple to follow and fast in its implementation, and its high accuracy and usability are demonstrated empirically on a practical data set with large varieties.