{'fraction_non_alphanumeric': 0.019102196752626553, 'fraction_numerical': 0.0, 'mean_word_length': 5.022988505747127, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Repairing Inconsistent Taxonomies Using MAP Inference and Rules of Thumb

Several authors have developed relation extraction methods for automatically learning or refining taxonomies from large text corpora such as the Web. However, without appropriate post-processing, such taxonomies are often inconsistent (e.g. they contain cycles). A standard approach to repairing such inconsistencies is to identify a minimally consistent subset of the extracted facts. For example, we could aim to minimize the sum of the confidence weights of the facts that are removed for restoring consistency. In this paper, we present MAP inference as a base method for this approach, and analyze how it can be improved by taking into account dependencies between the extracted facts. These dependencies correspond to rules of thumb such as "if a given fact is wrong then all facts that have been extracted from the same sentence are also likely to be wrong", which we encode in Markov logic. We present experimental results to demonstrate the potential of this idea.