{'fraction_non_alphanumeric': 0.030371203599550055, 'fraction_numerical': 0.0011248593925759281, 'mean_word_length': 4.59748427672956, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
New statistic in P-value estimation for anomaly detection

Given n nominal samples, a query point η and a significance level a, the uniformly most powerful test for anomaly detection can be to test p(η) ≤ α, where p(η) is the p-value function of η. In [1] a p-value estimator is proposed which is based on ranking some statistic over all data samples, and is shown to be asymptotically consistent. Relying on this framework we propose a new statistic for p-value estimation. It is based on the average of K nearest neighbor (K-NN) distances of η within a K-NN graph constructed from n nominal training samples. We also provide a bootstrapping strategy for estimating p-values which leads to better robustness. We then theoretically justify the asymptotic consistency of our ideas through a finite sample analysis. Synthetic and real experiments demonstrate the superiorities of our scheme.