{'fraction_non_alphanumeric': 0.011004126547455296, 'fraction_numerical': 0.0, 'mean_word_length': 5.678899082568807, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Efficient pattern discovery for semistructured data

The process of discovering frequent patterns from large semistructured data repositories is one of the hardest categories of tree mining problems, since it involves the discovery of unordered embedded tree patterns. Existing work has focused primarily on the discovery of ordered, induced trees. This work proposes a divide-and-conquer algorithm called WTIMiner to discover the complete set of frequent unordered embedded subtrees. The algorithm successfully reduces the complexity of pattern matching and counting problem that a regular tree mining algorithm faces. Experimental results demonstrate the efficiency and scalability of WTIMiner in terms of both time and space