{'fraction_non_alphanumeric': 0.01749271137026239, 'fraction_numerical': 0.0, 'mean_word_length': 5.560509554140127, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Knowledge-Guided Schema Discovery from Semi-Structured Text

A significant amount of information is expressed as the semi-structured, non-grammatical text found in auction listings and classified advertisements. There would be value in automatically fitting this type of information to relational database schemas. Work has been conducted on automatically populating such a database, and automatic schema mapping is a large area of research, but the problem of automatically generating such a schema is relatively unaddressed. Schema Discovery is the recent research thrust that addresses this. In this paper, we introduce the TESS system for knowledge-guided schema discovery from semistructured text. TESS performs term extraction over listing text and applies semantic reasoning to differentiate common terms into categories. This approach differs from existing schema discovery work in that it requires no hand-labeling or training examples, and it takes advantage of existing large databases of collected semantic knowledge.