{'fraction_non_alphanumeric': 0.02111801242236025, 'fraction_numerical': 0.0, 'mean_word_length': 5.06015037593985, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Question Answering using Common Sense Knowledge latent in Corpora and Utility Maximization Principle

In this paper, we propose two new methods for open-domain question answering. First, we use knowledge resembling “common sense” for question answering purposes. For example, the length of a runway in an airport must be a few kilometers, but a few centimeters. In practice, we use specific types of information latent in document collections to verify the correctness of each answer candidate. Second, we use the utility maximization principle to determine the appropriate number of answers for a list question. We estimate the expected value of the evaluation score, on the basis of the probability scores for multiple answer candidates. We show the effectiveness of our methods by means of experiments.