{'fraction_non_alphanumeric': 0.016181229773462782, 'fraction_numerical': 0.0, 'mean_word_length': 4.951923076923077, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Web Search Contextual Crawler Using Ontology Relation Mining

In order to increase the correctness and the recall of vertical web search system, a novel web crawler ORC is proposed in this paper. By introducing ontologies, the semantic concepts are defined. The relations between ontologies are exploited to evaluate the importance of web documents' context. These ontologies are organized by a network structure that is updated periodically. Thanking to the ontology relation mining, the crawling space is expanded and focused more. A primitive vertical search system is built based on ORC at the end of this paper.