{'fraction_non_alphanumeric': 0.01125703564727955, 'fraction_numerical': 0.0, 'mean_word_length': 5.239766081871345, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Organizing Web Documents into Thematic Subsets Using an Ontology

The volume and diversity of documents in the WWW, and the fact that these documents are connected to each other by links that bear important semantics generate requirements for effective organization of web content. This would enable effective browsing and searching. Both tasks are based on similarity between documents at the semantic level. In this context we propose a novel similarity measure between web documents, characterized by their incoming links, that can be used in conjunction with a clustering algorithm to discover groups of web documents on the same topic. Assuming each document is characterized by a sets of terms of an ontology, the similarity measure can compute a collective similarity between sets of terms seen as a whole, rather than between terms that would be independent. The similarity measure is embedded in different clustering algorithms and is extensively evaluated against real web document collections classified by human experts, in order to prove its efficiency.