{'fraction_non_alphanumeric': 0.0055248618784530384, 'fraction_numerical': 0.0, 'mean_word_length': 5.886075949367089, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Universal estimation of the optimal probability distributions for data compression of discrete memoryless sources with fidelity criterion

Output probability distributions of the test channels play important roles in data compression of discrete memoryless sources with fidelity criterion. In this paper a universal algorithm for estimating the output probability distributions is proposed. Sample size required by the algorithm is evaluated under a criterion of estimation similar to that of PAC learning in the computational learning theory.