{'fraction_non_alphanumeric': 0.03167420814479638, 'fraction_numerical': 0.0022624434389140274, 'mean_word_length': 5.020408163265306, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Semantic verification in an online fact seeking environment

Many artificial intelligence tasks, such as automated question answering, reasoning or heterogeneous database integration, involve verification of a semantic category (e.g. "coffee" is a drink, "red" is a color, while "steak" is not a drink and "big" is not a color). We present a novel algorithm to automatically validate a semantic category. Contrary to the methods suggested earlier, our approach does not rely on any manually codified knowledge but instead capitalizes on the diversity of topics and word usage on the World Wide Web. We have tested our approach within our online fact-seeking (question answering) environment. When tested on the TREC questions that expect the answer to belong to a specific semantic category, our approach has improved the accuracy by up to 14% depending on the model and metrics used.