{'fraction_non_alphanumeric': 0.01594896331738437, 'fraction_numerical': 0.0, 'mean_word_length': 4.709090909090909, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Question Classification using Latent Focus Words

In this paper we present a novel approach to the problem of selecting question focus words (QFWs) for the task of question classification. We show that by starting with a random choice of focus words, we can train a system that is comparable to one based solely on heuristics. The algorithms explored provide a more elegant approach to the problem, and one that is likely to scale better to larger data and finer question categories. The results show promise for exploring more avenues with such an approach, and systems that can integrate latent QFWs with strong heuristics. .