{'fraction_non_alphanumeric': 0.01405152224824356, 'fraction_numerical': 0.0, 'mean_word_length': 4.9375, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Scrubbing the Web for Association Rules: An Application in Predictive Text

Modern smartphones have led to an explosion of interest in predictive text. Predicting the next word that a user will type saves precious time on the compact keyboards that smartphones use. By leveraging the vast amounts of text data available on the Internet, we can easily gather information on natural human writing. We can then use this data with association rules to efficiently determine the probability of one word appearing after another given word. In this paper, we explore the gathering of text data from online social media. We also examine the use of association rules for predictive text, and develop an algorithm that can quickly and efficiently generate rules for predictive text. The results of the presented algorithm are compared to Google's Android keyboard.