{'fraction_non_alphanumeric': 0.01950585175552666, 'fraction_numerical': 0.0, 'mean_word_length': 4.923076923076923, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reliability, sampling, and algorithmic randomness

We investigate the relationship between software testing, statistical reliability modelling, and random sampling. Let P be a program whose reliability is assessed statistically based on the number of times P fails when executed on a sample S of its inputs. We show that if this assessment is grossly misleading, then either the degree of randomness (Kolmogorov complexity) of S is not high or the sum of the lengths of P and any program that satisfies the requirements for P is not much less than the size of the “population” of runs from which S was drawn. This implies that there is an a priors condition that, if satisfied, guarantees that a reliability assessment obtained by sampling cannot be grossly misleading.