{'fraction_non_alphanumeric': 0.011412268188302425, 'fraction_numerical': 0.0, 'mean_word_length': 5.622641509433962, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Lossless coding with generalized criteria

This paper presents prefix codes which minimize various criteria constructed as a convex combination of maximum codeword length and average codeword length, or, a convex combination of the average of an exponential function of the codeword length and the average codeword length. This framework encompasses as a special case several criteria previously investigated in the literature, while relations to universal coding is discussed. The coding algorithm derived is parametric resulting in re-adjusting the initial source probabilities via a weighted probability vector according to a merging rule. An algorithm is presented to compute the weighting vector.