{'fraction_non_alphanumeric': 0.039265281541704306, 'fraction_numerical': 0.016832279433905452, 'mean_word_length': 4.6907128169979435, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Practical Learned Lossless JPEG Recompression with Multi-Level Cross-Channel Entropy Model in the DCT Domain

JPEG is a popular image compression method widely used by individuals, data center, cloud storage and network filesystems. However, most recent progress on image compression mainly focuses on uncompressed images while ignoring trillions of already-existing JPEG images. To compress these JPEG images adequately and restore them back to JPEG format losslessly when needed, we propose a deep learning based JPEG recompression method that operates on DCT domain and propose a Multi-Level Cross-Channel Entropy Model to compress the most informative Y component. Experiments show that our method achieves state-of-the-art performance compared with traditional JPEG recompression methods including Lepton, JPEG XL and CMIX. To the best of our knowledge, this is the first learned compression method that losslessly transcodes JPEG images to more storage-saving bitstreams.


Introduction
JPEG [44], a popular image compression algorithm, is used by billions of people daily and JPEG images spread widely in data center, cloud storage and network filesystems. According to a survey, in operating network filesystems like Dropbox, JPEG images make up roughly 35% of bytes stored [21]. However, most of these images are not sufficiently compressed due to the limitation of JPEG algorithm: relying on hand-crafted module design and hard to eliminate data redundancy adequately. Actually, JPEG algorithm has been developed for many years so that it has already been outperformed by other more recent image compression methods, such as JPEG2000 [34], BPG [10], intra coding of VVC/H.266 [31] and deep-learning based methods [8,9,29]. However, these subsequent image compres-sion methods devote to process original images in lossless format like PNG while ignoring the need of further compressing trillions of existing JPEG images losslessly.
Considering the recompression needs of storage service, there exist several methods on further compression of JPEG images, e.g. Lepton [21], JPEG XL [6,7], and CMIX [1]. However, they rely on hand-crafted features and independently optimized modules, limiting compression efficiency. Along with the quick proliferation of mobile devices saving and uploading JPEG images, these storage systems have become gargantuan and existing JPEG recompression algorithms are not expected to be optimal and general solutions to storage challenges faced by service providers.
We propose an efficient JPEG image lossless recompression neural network using quantized DCT [5] coefficients as input, which is stored in the JPEG file. To the best of our knowledge, this is the first work proposing a deep learning based model dedicated for lossless recompression of JPEG images and outperforms existing traditional methods including Lepton, JPEG XL, and CMIX by a large margin.
In our method, JPEG in YCbCr 4:2:0 format is considered because of its popularity. As shown in Fig. 1, we first construct a color-space entropy model for YCbCr 4:2:0 format which extracts side information z as prior to build conditional distribution of the three components. Then we further exploit the correlation of Y, Cb, and Cr components sequentially (i.e. Cb component conditioned on Cr, Y component conditioned on both Cb and Cr). Additionally, since Y component is much more informative than Cb and Cr components, we propose a Multi-Level Cross-Channel (MLCC) entropy enhancement model for Y component to reduce the mismatch between estimated and true data distribution.
In conclusion, our main contributions include: • We propose an end-to-end lossless compression model for images already encoded with JPEG format. To the best of our knowledge, this is the first approach investigating learning-based JPEG recompression, which further benefits the widely adopted JPEG format based on powerful data-driven techniques.
• Experiments show that our proposed JPEG recompression method achieves state-of-the-art performance, outperforming Lepton, JPEG XL and CMIX. Also, our model has reasonable running speed and is a promising candidate for practical JPEG recompression.
2. Related work 2.1. Overview of JPEG algorithm JPEG algorithm first converts image from RGB sources to YCbCr color space (one luma component (Y) and two chroma components (Cb and Cr)). Then, considering human visual system is far more sensitive to brightness details stored in the luma component than to color details stored in two chroma components, the luma component is supposed to be more important than chroma components. Most JPEG images adopt YCbCr 4:2:0 format, where Y retains the same resolution while Cb and Cr components are subsampled as 1 4 of their original resolution. Next, every component is divided into 8×8 pixel blocks and each pixel block is transformed by discrete cosine transform (DCT) into a matrix of frequency coefficients (DCT coefficients) of the same size. Subsequently, these three components are quantized by two quantization tables: Y component uses a quantization table while Cb and Cr components share another  quantization table. Finally, all DCT coefficients are compressed by lossless Huffman coding [23]. Importantly, in order to deploy Huffman coding, the two-dimensional DCT coefficients have to be turned into a one-dimensional array. Zig-zag scanning is adopted here to group similar frequencies together to obtain better performance.

JPEG recompression methods
Lepton [21] proposed by Horn et al., which achieves 22% storage reduction after recompressing JPEG losslessly, mainly focuses on optimization of entropy model and symbol representations. Instead of using Huffman coding, Lepton uses more efficient arithmetic coding [45]. Further more, combining unary, sign and absolute value, the representation method in Lepton outperforms other encodings like pure unary and two's complement with fixed length. Lepton also deals with DC component by predicting it from AC components and storing the residual.
JPEG XL [6,7] is a versatile compression method supporting both lossless and lossy compression. For existing JPEG images, losslessly transcoding them to JPEG XL is also supported. JPEG XL achieves better compression ratio by extending the 8 × 8 DCT to variable-size DCT which allows block size to be one of 8, 16 or 32. Besides, JPEG XL uses Asymmetric Numeral Systems [15] in place of Huffman coding. Instead of using fixed quantization matrix globally, the quantization matrix in JEPG XL can be scaled locally to better accommodate the complexity in different areas. Compared with the primitive DC coefficient prediction mode in JPEG, JPEG XL supports eight modes and will choose the mode producing the least amount of error.
CMIX [1] is a general lossless data compression program aimed at optimizing compression ratio at the cost of high CPU/memory usage. It achieves state-of-the-art results on several compression benchmarks. CMIX uses an ensemble of independent models to predict the probability of each bit in the input stream. The model predictions are combined into a single probability using a context mixing algorithm. The output of the context mixer is refined using an algorithm called secondary symbol estimation (SSE). CMIX can compress all data files losslessly, including JPEG images.

End-to-end image compression
Learned lossy compression. Since Ballé et al. [8] proposes an end-to-end learned image compression method based on variational autoencoder [43] (VAE) architecture, subsequent deep-learning based approaches continue to explore and improve similar architectures (e.g. [9,12,13,18,19,22,27,29,36,46,47]). These methods initially focus on how to deal with non-differential quantization and rate estimation to enable end-to-end training [8,36]. Then, in order to build more accurate entropy model to further reduce the cross entropy (corresponding to bit rate), some methods [9,22] devote to introduce hyperprior models to the VAE architecture. More recent approaches investigate context models for more accurate entropy estimation, e.g. adding pixel-wise [29] or channel-wise autoregressive [30] modules. These techniques have greatly improved the performance of learned image compression. Actually, all of the learned methods mentioned above have outperformed JPEG. The performance of newest methods [18,47] even surpasses the intra coding of latest standard VVC/H.266 [31]. However, they focus on the compression of images stored in lossless format like PNG, serving as replacement of JPEG instead of recompressing existing JPEG images.
Learned lossless compression. Our research is more related to learned lossless image compression. In theory, any probabilistic model can be used together with entropy coders to compress data into compact bitstreams losslessly. The bit rate lower bound is given by the probabilistic model according to Shannon's landmark paper [37] (i.e. the more accurate the probabilistic model is, the lower the bit rate will be). Representative learned lossless image compression methods include likelihood-based generative models (e.g. PixelCNN [32], PixelRNN [42], MS-PixelCNN [35]), bitsback methods (e.g. BB-ANS [39], Bit-Swap [25], Hilloc [40]) and flow-based models (e.g. IDF [20], IDF++ [41], iVPF [49]). To reduce computation complexity, a parallelizable hierarchical probabilistic model is proposed in Figure 1. Overall architecture of the proposed JPEG lossless recompression method. AE and AD stands for arithmetic coding and arithmetic decoding respectively. Blue and green lines indicate data-flow for encoding and decoding respectively, orange lines are shared.
L3C [28], which is the first practical full-resolution learned lossless image compression method. This hierarchical probabilistic modeling idea is later improved by SReC [11] and the multi-scale progressive statistical model [48]. Nevertheless, same as learned lossy image compression, these learned lossless compression methods still only consider images stored in PNG format while ignoring the vast already existing JPEG images. In our investigation, we find these methods cannot be used directly to losslessly compress JPEG images, which we focus on in this work.

Framework
The overall framework of the proposed model is presented in Fig. 1. Since DCT domain is used in our method to design an efficient entropy model, we first rearrange each 8 × 8 block of DCT coefficients in order to learn better distribution (Sec. 3.2). Because JPEG usually adopts YCbCr 4:2:0 format, we apply a Coefficient Fusion Model (CFM) detailed in Sec. 3.3 to align the shape of DCT coefficients from different color components. After shape alignment, DCT coefficients are sent to a Hyper Encoder and will produce the hyperprior z, which is saved in the bitstream as side information. Subsequently, the coding prior of the three color components will be obtained after the hyperprior z goes through Hyper Decoder and a Coefficient Prior Split Model (CPSM) detailed in Sec. 3.3.
Besides the shared hyperprior z, we further reduce the statistical redundancy by explicitly modelling the correlation between color components and DCT coefficients. Detailed in Sec. 3.3, we estimate Cr distribution conditioned on z, Cb distribution conditioned on both z and Cr, and Y distribution conditioned on z, Cr and Cb. According to human perception, Y component contains more information than Cb and Cr components. We propose a multi-level cross-channel (MLCC) entropy enhancement model to better predict Y distribution, which is described in Sec. 3.5.
Finally, we use arithmetic coding [45] based on these probabilistic distributions to compress component coefficients into compact bitstreams losslessly.  JPEG encoder transforms the pixels in an 8 × 8 block to a matrix of DCT coefficients in the same size and each coefficient in the matrix stands for one frequency. The top left corner of this matrix is the DC component, while the remaining 63 coefficients are AC components. As shown in Fig. 2, we first adopt the same way as [17] to rearrange DCT coefficients so that the same frequency from all blocks are extracted together to form the spatial dimension, and different frequencies form the channel dimension. This operation converts Y, Cb, Cr components to 64 channels with 1 64 of their original spatial size. A lot of coefficients in AC components will go to zero after quantization. Therefore, we rearrange the channel dimension by zigzag scan in the original DCT matrix to make zero values as close as possible to exploit the structural information.

Cross-Color Entropy Model
Cross-color correlation can be modeled both implicitly (through shared hyperprior) and explicitly (through the entropy parameters network).
Hyperprior network proposed in [9] can be viewed as an efficient entropy model which generates hyperprior z as side information and then produces the scale parameters for Gaussian distribution conditioned on z. This method is improved in their later work [29], where hyperprior is combined with context-based predictions. The same hypernetwork as [29] is used in our method to extract hyperprior from fused color components, which serves as side information and models the cross-color correlation implicitly. However, this VAE-like model is unable to support YCbCr 4:2:0 format directly due to different spatial resolutions. Similar to [16], we add Coefficient Fusion Model (CFM) and Coefficient Prior Split Model (CPSM) to Hyper Encoder and Hyper Decoder respectively. Architecture of CFM is shown in Fig. 3a, through which the three color components are reshaped and fused. As shown in Fig. 3b, CPSM is used to split the prior of the three color components, producing Y prior , Cb prior and Cr prior .
Each element of DCT coefficients is modeled as a single Laplace distribution with its own scale b and location µ parameters. We split Cr prior into b cr and µ cr to obtain Laplace parameters of Cr component. b cr and µ cr have the same size as Cr component. As formulated in Eq. (1), the probability of Cr given z is calculated in a factorized manner. Subsequently, Cr component is fed to Entropy Parameters network (Fig. 3c) as context of Cb component and is fused with Cb prior . The output of this model is split into b cb and µ cb . As a result, the probability mass function (PMF) of Cb component will be conditioned on both Cr component and hyperprior z and is shown in Eq. (2).
Cb and Cr components are upsampled by 3 × 3 stride-2 transposed convolution and concatenated to serve as context of Y component. They are fed together with Y prior to Entropy Parameters network and we can obtain hyper y (Fig. 1) to calculate the conditional distribution p y| z,c b ,cr (y| z, c b , c r ). Nevertheless, this PMF similar to Cr and Cb components is not powerful enough for the most informative Y component. In the following section, we propose a more suitable context modelling method to further reduce redundancy in Y component.

Matrix Context Model
Context modeling is an efficient technique to predict precise probabilistic distribution of unknown symbols based on adjacent symbols that have already been decoded. Previous learning-based codecs adopt a spatially auto-regressive context model, which requires decoding each symbol sequentially. While these methods are effective, they are impractical for real-world deployment due to low computational efficiency caused by the lack of parallelization [24,29]. Then, channel-conditional (CC) context models are explored in [30], which splits the symbol tensor along channel dimension into many equal-size slices and each slice can be modeled conditioned on all already decoded slices, providing much better parallelism. Subsequently, He et al. [19] proposes a novel spatial parallel context model with a two-pass decoding approach, where the symbol tensor is decomposed into two groups according to checkerboard pattern and then one group serves as context of the other to build conditional distribution. Meanwhile, a hierarchical probabilistic modeling idea, which is similar in spirit to spatially parallel context model, is prevalent in learned lossless image compression. Hierarchical models [11,28,48] downsample an input image into different low-resolution representations and the probabilistic distribution of the input image is the product of the conditional distributions in multiple scales.
In this paper, we propose a novel parallelizable matrix context model to enhance the entropy estimation of Y component. As shown in Fig. 4, we first use space-to-depth operation (i.e. inverse operation of PixelShuffle [38]) to convert Y component to Y ′ (64 × 4 channels with 1 4 of the original spatial size). Then we partition Y ′ along channel dimension into 4 equal-size slices (i.e. 4 rows in the matrix representation, Y ′ = r (1) r (2) r (3) r (4) ), where each row is modeled conditioned on all previously decoded rows. Therefore, the conditional distribution p y| z,c b ,cr (y| z, c b , c r ) can be calculated as: Each row has 64 channels and there exists considerable redundancy. Hence, each row is further partitioned to explicitly exploit this channel-wise correlation, i.e. r (i) = where n is the number of split columns at row i. Let R (i) = r (i−1) , · · · , r (1) , z, c b , c r denote the context and prior for r (i) (specifically, where c (i) j is column j at row i, n is the number of columns at row i, and i = 1, 2, 3, 4.
Let C j−1 , R (i) denote the context and prior for column j at row i (specifically, C where y (i) jk is coefficient k in column j at row i, m j is the number of coefficients in column j, i = 1, 2, 3, 4, j = 1, 2, · · · , n, and k = 1, 2, · · · , m j .
According to the rearrangement in Sec. 3.2, the 64 channels at each row in our matrix context model represent different frequency, where higher frequency has been quantized more aggressively and contains less information. Therefore, we reverse the channel order in each row when formulating the matrix context (i.e. c (i) 1 in Fig. 4 is the AC coefficients representing the highest frequency in r (i) ). Moreover, we design non-uniform partition for the column dimension to balance this information asymmetry. Specifically, we let the number of columns be n=9, the lengths of each column (c (i) j , j = 1, 2, · · · , 9) are set as 28, 8, 7, 6, 5, 4, 3, 2 and 1 respectively.

Multi-Level Cross-Channel Entropy Model
A deep neural network named Multi-Level Cross-Channel (MLCC) is designed to implement our matrix context entropy model to estimate Laplace parameters (location   µ and scale b) in Eq. (5), where we interpret the matrix context model as cross-channel autoregressive model: autoregression along the rows is viewed as Outer Channel (top in Fig. 5) to generate prior of each row (i.e. pri (i) in Fig. 5), autoregression along the columns in each row is modeled as Inner Channel (bottom left in Fig. 5) to generate Laplace parameters for each column. MLCC leverages both matrix context and hyper y (Sec. 3.3) to learn more powerful PMF for the most informative Y component.
As shown in Fig. 5, We first adopt space-to-depth to rearrange hyper y into h ′ as prior of Outer Channel (h ′ has 64 × 4 channels with 1 4 spatial size of hyper y ). Meanwhile, Y component is reshaped by space-to-depth and then split into 4 rows (Sec. 3.4), the first row r (1) is predicted conditioned solely on h ′ . However, unlike [30], this step in our method will generate prior pri (i) for current row r (i) rather than entropy parameters. Next, the current row and its own pri (i) are sent to Inner Channel. In Inner Channel model, the row is partitioned into n columns (we set n to 9 in our method). And then the first column is compressed using a single Laplace entropy model with mean and scale conditioned only on pri (i) , while the entropy model for the remaining columns (e.g. c (i) j ) are conditioned on pri (i) and all decoded coefficients in previous columns (e.g. c (i) 1 , · · · , c (i) j−1 ). After all columns in current row are decoded by Inner Channel model, they will be concatenated with h ′ and all previously decoded rows, and then processed by Outer Channel model again to generate prior pri (i+1) for next row r (i+1) . Repeat this operation until all rows in Y component is encoded or decoded. With this MLCC model, decoding will be slower than encoding because of the conditional relationship. In decoding stage those columns and rows must be decoded sequentially. However, all the coefficients in the same column can be processed in parallel, ensuring that the overall sequential complexity is constant (irrelevant of the input image size), which is 4 × n (we let the number of columns be n=9). This guarantees that our method can be practical for recompressing high resolution JPEG images.

Loss Function
The expected code length arithmetic coding [45] can achieve, using our learned distribution as its probability model, is given by the cross entropy: where p is the true distribution of DCT coefficients, p is estimated by entropy model. Our model is trained to minimize this cross entropy to minimize the bit-length.

Settings
Datasets. The training dataset comprises the largest 8000 images chosen from the ImageNet [14] validation set, where each image contains more than one million pixels. Similar to [8,9,19], each image is disturbed by uniform noise and downsampled. We evaluate our model on four datasets: Kodak [26] dataset with 24 images, 100 images chosen from DIV2K [4], CLIC [3] professional test dataset with 250 images and CLIC mobile dataset with 178 images. Since our method processes images entirely in DCT domain, before fed to model, we use torchjpeg.codec.quantize at quality [2] to extract quantized DCT coefficients with given JPEG quality level, which guarantees that the result is the same as using JPEG images generated from image libraries like Pillow. We fix the quality level of the training dataset at 75 if not specified.
Implementation details. During training, 256 × 256 pixel patches are randomly cropped from training data and then quantized DCT coefficients are extracted. Our model is implemented in PyTorch [33] and we adopt Adam optimizer. The batch-size is 16 and the learning-rate is 10 −4 . We apply gradient clipping for the sake of stability and train the model for 2000 epochs. All the speed testing results are obtained on single Nvidia GeForce GTX 1060 6GB (GPU) for learned methods and Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (CPU) for non-learned ones.

Performance
Performance comparison with other JPEG recompression methods. We compare the proposed model against other state-of-the-art methods for JPEG recompression on four test datasets mentioned in Sec. 4.1. We adopt our best model with 9 non-uniform channel slices, where the number of channels is split as [28,8,7,6,5,4,3,2,1]. As shown in Tab. 1, with quality level set as 75, our method achieves lowest bit rate on all evaluation datasets and obtains about 30% compression savings. Our method is much faster than CMIX but slower than JPEG XL and Lepton. However, it is worth noticing that our implementation of arithmetic coder is naive and our model has not been optimized to achieve the fastest speed.
Performance on different quality levels. We test our models on Kodak with 7 different JPEG quality levels (i.e. quality = 35, 45, 55, 65, 75, 85, 95). The results are presented in Fig. 6. It shows that our method still outperforms other methods, which shows that our model trained for quality = 75 can generalize well to different quality levels except very high quality like 95. More detailed results are given in the appendix.
Performance comparison with other learned lossless compression methods. We compare our method with rep-resentative learned lossless image compression methods designed for PNG images, including IDF [20] and multi-scale model [48]. These methods are designed for RGB 4:4:4 format, so we convert JPEG 4:2:0 input data to RGB 4:4:4 format by upsampling Cb and Cr components. This upsampling operation increases resolution and may cause unfair comparison. Consequently, we also carry out experiments with JPEG 4:4:4 source format and convert it to RGB 4:4:4 as model input. By modifying these models slightly, these methods can also deal with JPEG 4:2:0 format directly, we present results of this kind of experiments in the appendix.
As shown in Tab. 2, both for JPEG 4:2:0 and JPEG 4:4:4 format, our models outperform IDF and multi-scale model by a large margin. Additionally, we evaluate neural network latency of our model, L3C [28], IDF [20] and multi-scale [48] in Fig. 7, which shows our model is faster.

Ablation study
We test a serial of models on Kodak with quality level 75 to investigate the effect of cross-color entropy model, multi-level cross-channel entropy enhancement model and non-uniform channel slices. Effectiveness of cross-color entropy model. To verify the effectiveness of our cross-color entropy model, we compare three models. Cross-color case 1: Y, Cb, and Cr components are modeled totally independent of each other, i.e. there are three side information z Y , z Cb , z Cr , and Y (with MLCC), Cb and Cr components are dependent on z Y , z Cb , z Cr respectively. Cross-color case 2: Hyperprior z is shared by Y, Cb, and Cr components, and these three color components are conditioned on the shared z except that the entropy model of Y component is enhanced by MLCC. Here z provides implicit cross-color correlation while no explicit modeling is used. Cross-color case 3: Three color components are modeled totally independent of each other, while they are treated equally and are all modeled by independent hyperprior with corresponding MLCC module. We give detailed architectures in the appendix. As shown in Tab. 3, savings of case 1 and case 2 are lower than our proposed model and case 2 is better than case 1, indicating that both implicit and explicit cross-color correlation modeling is helpful to bit saving. Although the network capacity and parameter number are the largest in case 3, its performance is still slightly worse than our model, proving the effectiveness of our cross-color entropy model. Effectiveness of MLCC model. To verify the effectiveness of MLCC, we replace MLCC with three different models while keeping other parts of the model unchanged. Only Outer Channel drops the Inner Channel module in MLCC, which means there are no column split operation. Only Inner Channel has no space-to-depth and row split operations, which only has Inner Channel module in MLCC. Column-to-row is a variation of our rowto-column MLCC, which adopts column split first and then row split. Details about these models are given in the appendix. Shown in Tab. 3, the above three replacements of MLCC deteriorate the compression savings, which verifies the effectiveness of MLCC.
Effectiveness of non-uniform slices. We compare two models to verify that non-uniform column slicing in MLCC is more effective for JPEG recompression. Uniform 8 slices divides rows evenly into 8 columns, while non-uniform 8 slices divides rows into 8 columns of size [36,7,6,5,4,3,2,1] respectively. As demonstrated in Tab. 3, non-uniform 8 slices model has the same column number as uniform 8 slices model but its compression ratio is about 1.5% higher.

Conclusion
We propose a novel Multi-Level Cross-Channel entropy model for lossless recompression of existing JPEG images, which achieves state-of-the-art performance on Kodak, DIV2K, CLIC.mobile and CLIC.pro and has reasonable running speed. We also show that our method trained with quality level 75 can generalize well on other quality levels except very high quality like 95. To the best of our knowledge, this is the first learned method targeting lossless recompression of JPEG images. For future work, we will explore the generalizability on very high quality levels. As shown in the left of Fig. 1, the Y component occupies much more bitrates than Cb and Cr components. Then we analyze the entropy of each channel in the Y component and find that channels with smaller indexes (corresponding to higher frequency) occupy less entropy, which demonstrates the information asymmetry mentioned in Sec. 3.4 in the main text.

Entropy coding and probability model
Lossless compression technique is essentially an entropy coder along with a probability model. JPEG algorithm adopts Huffman coding together with Huffman tables defining the probability model to compress quantized DCT coefficients losslessly. However, these coefficients are considered independently and identically distributed (i.i.d.) under this fixed probability model, resulting in a mismatch between estimated and actual data distribution, which decreases compression savings.
Our method contains two significant improvements in this aspect. First, our method uses Laplace distribution with different parameters for each coding symbol to obtain an adaptive probability model, where the two parameters (scale b and location µ) of each Laplace distribution are learned by neural networks. It is known that AC coefficients of Fourier-related transformations, like DCT coefficients of JPEG algorithm, obey Laplace distribution [2]. Second, we replace the Huffman coding with arithmetic coding, which is a more efficient technique that almost achieves the lower bound entropy for long enough symbol streams.

Details about performance on different quality levels
Tab. 1 is the detailed data corresponding to Figure 6 in the main text. All results at row Ours (QP 75) and row Ours (QP 95) are obtained by our models trained with QP 75 and QP 95 respectively. While results in row Ours (QP independent) are generated by 7 models trained with QP 35,45,55,65,75,85 and 95 respectively to ensure that QP value is exactly the same for training and testing. Data of these three rows are visualized by Fig. 2. It shows that the Ours (QP 75) has comparable performance with Ours (QP independent) at all QPs lower than 85, which further demonstrates that our model trained for quality = 75 can generalize well to different quality levels except very high quality like 95.

Details about comparison with learned lossless methods
We reproduce the multi-scale model following the instructions in the original paper, and our reproduced model achieves bits per sub-pixel (BPSP) of 3.942 (computed by negative log-likelihood) on ImageNet64 dataset, which is slightly better than BPSP 3.96 presented in the original paper.
We also reproduce IDF based on the source code released by the authors. Nevertheless, the original IDF can only accept input with fixed size due to the logistic mixture model (LMM) used for latent prior z L (L is the level of flows). To make IDF resolution-adaptive, we replace the LMM with the univariate non-parametric density model used in [1]. Our reproduced model achieves BPSP of 3.879 (computed by negative log-likelihood) on ImageNet64 dataset, slightly better than BPSP of 3.90 given in the original paper.
As stated in the main text, these two methods are designed for RGB 4:4:4 input, so we convert JPEG 4:2:0 input data to RGB 4:4:4  For IDF and multi-scale model using DCT coefficients as input, we do not adopt the DCT coefficients rearrangement proposed in the main text to avoid enormous architecture modification (this is because they are originally designed for thin input format like RGB values with only 3 channels). To deal with DCT 4:2:0 input, we reshape Y component to 4 channels by space-to-depth operation and then concatenate these 4 channels with Cb and Cr components to form 6-channel inputs with 1 4 of their original resolutions. Meanwhile, both of the methods need to be adjusted slightly to fit this kind of input. For multi-scale model, the number of input channels is changed from 3 to 6, and autoregression over RGB channels is disabled. For IDF, we start from the original IDF and increase the number of input channels to 6.
For all of these experiments, we use Adam optimizer and MultiStepLR scheduler. We use the same training set and use Kodak Fig. 4, Fig. 5 and Fig. 6 show the architectures for the three cases of cross-color entropy model respectively.

Variants of MLCC model
This section shows architecture details of the three variants of MLCC (i.e. Only Outer Channel in Fig. 8, Only Inner Channel in Fig. 10, and Column-to-row in Fig. 12) and their corresponding context models (Fig. 7, Fig. 9, Fig. 11) mentioned in Sec. 4.3 in the main text.   Fig. 7 is its corresponding context model. Letting n, m representing the channel number of input tensor and next slice to be modeled respectively, C1, C2 and C3 are decided by C1 = n − d, C2 = n − 2 * d, C3 = 2 * m, d = (n − 2 * m)//3. Blue and green lines indicate data-flow for encoding and decoding respectively, orange lines are shared.

Row Split
Row Split Row Split r (1) r (2) r (3) r (4) ··· Space-to-depth Space-to-depth Space-to-depth ··· Figure 11. Matrix context modeling method in column-to-row manner.Solid arrows indicate data operation and dotted arrows denote conditional relationships. Light grey and light blue dotted arrows align with Outer Channel and Inner Channel in Fig. 12