{'fraction_non_alphanumeric': 0.0196078431372549, 'fraction_numerical': 0.0, 'mean_word_length': 5.3828125, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Empirical Kolmogorov Complexity

The Kolmogorov complexity of a string is the shortest program that outputs that string, and, as such, it provides a deterministic measure of the amount of information within the string that is related, but independent of, Shannon entropy. In practice, this complexity measure is uncomputable and mainly useful for deriving theoretical bounds. In this work, we consider the task of empirically computing Kolmogorov complexity for short strings. We design and build a virtual One Instruction Set Computer that we use to empirically compute the Kolomogorov complexity of short strings (within this computing model). It is hoped that this approach could inform novel cloud-based data compression methods that more effectively digest human data than standard Lempel-Ziv based compression.