{'fraction_non_alphanumeric': 0.017735334242837655, 'fraction_numerical': 0.0, 'mean_word_length': 5.220338983050848, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Test of Relative Similarity

A statistical test of relative similarity is introduced: given independent samples from a target distribution and two candidate distributions, the test determines whether one of the candidate distributions is significantly more similar to the target than the other. The maximum mean discrepancy (MMD) is used as the metric for ranking relative similarity of the distributions, and a powerful, low-variance test is derived based on the joint asymptotic distribution of the MMDs between each target-candidate pair. The purpose of the test is to compare generative models , and to determine which of two models generates samples that are significantly closer to a real-world reference dataset of interest.