{'fraction_non_alphanumeric': 0.015223596574690771, 'fraction_numerical': 0.0, 'mean_word_length': 5.45398773006135, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Bringing Summarization to End Users: Semantic Assistants for Integrating NLP Web Services and Desktop Clients

We present PathSum, a high-performing hierarchical-topic based singleand multi-document automatic text summarization framework. This approach leverages Bayesian nonparametric methods to model sentences as paths through a tree and create a hierarchy of topics from the input in an unsupervised setting. We describe the generative model used to learn a topic tree based on hierarchical latent Dirichlet allocation, and an efficient converging algorithm that matches a built-up summary to the theme reflected in the input. We then illustrate how this method encompasses a framework that is amenable to generic and query-focused summarization, and even document creation. We evaluate our method on DUC and TAC data to compare it to standard multi-document news summarization with encouraging results, and in addition we conduct experiments on legal decision summarization to exemplify the generic ability of our method across different domains.