{'fraction_non_alphanumeric': 0.01833810888252149, 'fraction_numerical': 0.002292263610315186, 'mean_word_length': 5.442804428044281, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Arithmetic Coding for Length-constrained Joint Source Compression and Error Detection

Arithmetic coding (AC) is an entropy achieving lossless compression algorithm, which efficiency motivated its integration in emerging image and video compression standards like JPEG2000 and HEVC. However, this compression efficiency comes at a cost of a very sensitive compressed bitstream, not convenient when transmission over lossy channels is considered. This motivated researchers to propose the use of error detection and correction schemes combined with joint source channel coding for AC. In this paper, we change the paradigm and propose to use the arithmetic codes for error detection too based on length constraints. In fact, the starting observation is that the arithmetic encoded stream is very sensitive to errors, and tends to have a random behavior when errors are introduced. This random behavior induces, in general, a length change after arithmetic decoding which can be simply used to detect errors. The idea is very simple and can improve the decoding system quality when combined with retransmission strategies or appropriate error correction mechanisms, without adding any extra redundancy. In fact, the compression efficiency of the AC remains untouchable. This paper investigates the length-constrained error detection performance of AC with different source and errors configurations. The error detection efficiency is demonstrated and, unlike the commonly used Cyclic Redundancy Check (CRC) codes, the proposed method is more efficient when the number of errors is high. Finally, we present a review of the Chase-like arithmetic decoder that exploits the length-based error detection capacity of AC to make source error correction.