{'fraction_non_alphanumeric': 0.02122347066167291, 'fraction_numerical': 0.0, 'mean_word_length': 5.683333333333334, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Enriching Information Retrieval with Reading Level Prediction

The ability of a user to understand a document would seem to be an critical aspect of that document’s relevance, and yet a document’s reading difficulty is a factor that has typically been ignored in information retrieval systems. In this position paper we advocate for incorporating estimates of reading proficiency of users, and reading difficulty of documents, into retrieval models, representations for learning algorithms, and large-scale analyses of information retrieval systems and users, particularly for Web search. We describe key research problems such as estimating user proficiency, estimating document difficulty, and re-ranking, and summarize some potential future extensions that could exploit this new type of meta-data.