{'fraction_non_alphanumeric': 0.01132852729145211, 'fraction_numerical': 0.0, 'mean_word_length': 5.394736842105263, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Data structures and tools for document database generation: an experimental system

This paper is a contribution to the discussion of the structure and the elements of databases for document analysis tasks and the tools needed for database creation. It is pointed out that it is desirable to have a uniform document database that allows access to different kinds of data for different sub tasks within a complete document analysis system. Conceptual ideas pertaining to the data structure are discussed on the assumption of a hierarchical document structure. A description of an implemented data structure: is also included that may serve as a starting point for further investigation and discussion. Finally, we present INSEGD, an experimental system for interactive segmentation and labelling of arbitrary documents, which is still under development along with a tool box for automatically and semi-automatically generating segmentations for support in data generation.