{'fraction_non_alphanumeric': 0.0128, 'fraction_numerical': 0.0, 'mean_word_length': 5.731182795698925, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Vocabulary spectral analysis as an exploratory tool for scientific Web intelligence

A key scientific Web intelligence goal is to produce visualizations of academic Web spaces in order to reveal subject structures and identify relationships between different fields. We introduce an exploratory technique, vocabulary spectral analysis, designed to build intuition to help design effective procedures for clustering. We apply VSA to New Zealand University Web sites. The results suggest that subject-based academic Web site clustering is possible but requires extensive data filtering to give effective subject-based clusters.