{'fraction_non_alphanumeric': 0.025559105431309903, 'fraction_numerical': 0.0, 'mean_word_length': 5.064516129032258, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Reducing the Length of Shannon-Fano-Elias Codes and Shannon-Fano Codes

In some applications, both data compression and encryption are required. If the cryptanalyst knows the code construction rule and the probability mass function (PMF) of the source, then Huffman code provides no ambiguity, but Shannon-Fano-Elias coding is a good candidate since the order of symbols can be arbitrary in the encoding process, which produces exponential complexity for deciphering. However, the conventional Shannon-Fano-Elias code has relatively large expected length which makes it inefficient. In this paper, we propose two simple algorithms to reduce the expected length of Shannon-Fano-Elias code so that its efficiency is improved. This is achieved by first utilizing some proper PMF instead of the actual PMF in encoding, and then performing a codeword truncation. By applying the same strategy we can also reduce the length of Shannon-Fano codes