{'fraction_non_alphanumeric': 0.01056338028169014, 'fraction_numerical': 0.00528169014084507, 'mean_word_length': 5.252747252747253, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An efficient searching algorithm for approximate nearest neighbor queries in high dimensions

We present an approximate nearest neighbor search algorithm that uses heuristics to decide whether or not to access a node in the index tree, based on three interesting data distribution properties. We demonstrate that the proposed algorithm significantly reduces the number of nodes accessed over the algorithms that have been proposed in earlier works. Also, it is demonstrated that the proposed algorithm can retain close to 100% of the K nearest neighbors in most cases.