{'fraction_non_alphanumeric': 0.024886877828054297, 'fraction_numerical': 0.0015082956259426848, 'mean_word_length': 5.601990049751244, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Searching Social Updates for Topic-centric Entities

With the growing popularity of social networking services, real time short messages, such as Facebook news feeds and Twitter tweets, are becoming increasingly important information sources. People use these services to search for and consume content about interesting topics and events. Given a keyword search for a certain topic, simply returning those messages often does not give a comprehensive summary of the topic, primarily due to the brevity and redundancy of the messages. To address this challenge, we propose a topic centric entity extraction system where interesting entities pertaining to a topic are mined and extracted from short messages returned as search results on the topic. Specifically, we leverage signals from three main aspects: message content, social connections (i.e., message sender’s follower network), and referenced Web pages (i.e., URLs embedded within the messages), and propose: 1) page ranking algorithms for identifying relevant pages embedded within the messages; and 2) entity ranking algorithms for identifying relevant entities extracted from those URLs. Comprehensive experiments using real Twitter data show that our ranking algorithms are efficient and outperform baseline algorithms significantly in terms of extraction quality.