{'fraction_non_alphanumeric': 0.013186813186813187, 'fraction_numerical': 0.0, 'mean_word_length': 5.705882352941177, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Algorithms for Determining and Labelling Approximate Hierarchical Self-Similarity

We describe an algorithm for finding approximate sequence similarity at all scales of interest, being explicit about our modelling assumptions and the parameters of the algorithm. We further present an algorithm for producing section labels based on the sequence similarity, and compare these labels on some expert-provided ground truth for a particular set of recordings.