{'fraction_non_alphanumeric': 0.016536964980544747, 'fraction_numerical': 0.0, 'mean_word_length': 5.86, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Common Framework for Developing Table Understanding Models

A wealth of knowledge is contained in tabular data, and there are a vast number of efforts to model and capture this knowledge. Unfortunately, these efforts have disparate inputs, outputs, and goals hindering research progress and making table understanding tools difficult to use in practice. In this paper, we propose a table understanding framework that formalizes the problem of understanding tabular data into three distinct subtasks: cell classification, block detection, and relation prediction. We introduce a common API for table understanding systems that supports a host of existing approaches and allows easy development of new approaches. Our framework supports approaches that range from heuristic rules to probabilistic models, allows outputs that span simple, correlational tuples to sophisticated, semantic knowledge graphs, and provides tools for visualizing model outputs and transforming complex tabular data into flattened relational dataframes.