{'fraction_non_alphanumeric': 0.019696969696969695, 'fraction_numerical': 0.0, 'mean_word_length': 5.009090909090909, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Advances in News and Blog Analysis with Lydia

Lydia is a system for online text analysis currently focusing on news, blogs and Medline abstracts. The goal of the project is to build a relational model of people, places, and things through natural language processing of the input text and the statistical analysis of entity frequencies and juxtapositions. In this paper we describe our recent development of some new text analysis features of Lydia. This includes sentiment analysis in foreign language content, extraction of relations between entities and detection of “news storms”, i.e. spikes in popularity of entities caused by a common underlying event.