{'fraction_non_alphanumeric': 0.029227557411273485, 'fraction_numerical': 0.0041753653444676405, 'mean_word_length': 4.925925925925926, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Modeling with the Trace Norm

We have two goals: (1) to find a low-dimensional representation of text that allows generalization to unseen data, and (2) to group documents according to their similarities. Clearly these tasks are related—similar documents can be represented more compactly than dissimilar documents and vice versa. First, we discuss a model for text that smoothly penalizes the rank of the parameter matrix. Then, we discuss how to apply this model to clustering.