{'fraction_non_alphanumeric': 0.02230843840931135, 'fraction_numerical': 0.0, 'mean_word_length': 5.179640718562874, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Concept-Aware Ranking: Teaching an Old Graph New Moves

In ranking algorithms for Web graphs, such as PageRank and HITS, the lack of attention to concepts/topics representing Web page content causes problems such as topic drift and mutually reinforcing relationships between hosts. This paper proposes a novel approach to expand the Web graph to incorporate conceptual information encoded by links (anchor text) between Web pages. Using Web graph link structure and conceptual information associated with each Web page (automatically extracted from anchor text of Minks), a new graph is defined where each node represents a unique pair of a Web page and concept associated with that Web page, and an edge represents an explicit or implicit link between two such nodes. This graph captures inter-concept relationships, which is then utilized by ranking algorithms. Our experimental results show that such an approach improves accuracy (e.g., first X precision) by retrieving links which are more authoritative given a user's context