{'fraction_non_alphanumeric': 0.016210739614994935, 'fraction_numerical': 0.002026342451874367, 'mean_word_length': 4.488888888888889, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using XPaths of inbound links to cluster template-generated web pages

Template-generated Web pages contain most of structured data on the Web. 
 Clustering these pages according to their template structure is an important 
 problem in wrapper-based structured data extraction systems. These systems 
 extract structured data using wrappers that must be matched to only 
 particular template pages. Selecting single type of template from all crawled 
 Web pages is a time consuming task. Although there are methods to cluster Web 
 pages according to their structural similarity, however, in most cases they 
 are too computationally expensive to be applicable at Web-Scale. We propose a 
 novel highly scalable approach to structurally cluster Web pages by employing 
 XPath addresses of inbound inner-site links. We demonstrate the effectiveness 
 of our method by clustering more than one million Web pages from many real 
 world Websites in a few minutes and achieving >90% accuracy.