{'fraction_non_alphanumeric': 0.014061207609594707, 'fraction_numerical': 0.0, 'mean_word_length': 5.080402010050252, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Structure based semantic measurement for information filtering agents

With the volume of information on the Internet growing at an exponential rate, the needs of users to have their search results effectively filtered is increasingly important. A problem with most of the current search engines is that they only search on the specified keyword, which may be present in only a limited number of pages. This paper examines how a tree threshold function can be used in an information filtering agent (IFA) to extend the original keyword search to cover other related words within the domain, creating a keyword weighted semantic tree. The examination in this paper also considers how the metrics of the tree structure (shape, size, weights) influence the choice of related words for use in the extended search and what advantage this has over traditional methods. Further, that using a reduced word tree, which has been pruned using the tree pruning algorithm produces a significant increase in the number of profitable results for the user. Using these factors the analysis demonstrates equal accuracy to the benchmark comparison IFA but with increased efficiency and only a slight increase in execution time.