{'fraction_non_alphanumeric': 0.03462905630088293, 'fraction_numerical': 0.008198601077858044, 'mean_word_length': 4.644983818770227, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Constellation Design for Deep Joint Source-Channel Coding

Deep learning-based joint source-channel coding (JSCC) has shown excellent performance in image and feature transmission. However, the output values of the JSCC encoder are continuous, which makes the constellation of modulation complex and dense. It is hard and expensive to design radio frequency chains for transmitting such full-resolution constellation points. In this paper, two methods of mapping the full-resolution constellation to finite constellation are proposed for real system implementation. The constellation mapping results of the proposed methods correspond to regular constellation and irregular constellation, respectively. We apply the methods to existing deep JSCC models and evaluate them on AWGN channels with different signal-to-noise ratios (SNRs). Experimental results show that the proposed methods outperform the traditional uniform quadrature amplitude modulation (QAM) constellation mapping method by only adding a few additional parameters.


I. INTRODUCTION
In traditional communication systems, source coding and channel coding are two separate steps. Source coding is used to remove source redundancy, while channel coding is used to add redundant information to enhance robustness. Shannon's separation theorem proves that this two-step approach is optimal theoretically in the asymptotic limit of infinitely long source and channel blocks [1]. However, previous research proves that joint source-channel coding (JSCC) outperforms the separate source coding and channel coding in the finite blocklength reigme [2].
Nowadays, with the development of deep learning (DL) and convolutional neural networks (CNNs) [3], some CNNbased JSCC models are proposed to transmit images and deep intermediate features. In [4] and [5], CNN-based autoencoders (AEs) were used as the transmitter and receiver for the images. Images are encoded and transmitted at the transmitter, and the transmitted signals are received and reconstructed at the receiver after passing through noisy channels. Based on [4] and [5], [6] added a feedback model to improve the performance and robustness. Considering the change of SNR, [7] proposed a SNR-adaptive image transmission model. By [8], where the deep model can be split into the mobile device and the edge server to balance the computational load, [9] proposed BottleNet++ to compress and transmit the intermediate feature of the ResNet [10] over noisy channels. To improve the computing capability of the receiver, [11] proposed an asymmetrical AE-based JSCC model for transmitting the deep features of the VGG network [12]. The AE-based JSCC model was also applied to person re-identification (re-ID) [13]. All of these models are single-task learning (STL) models, and [14] proposed a multi-task learning (MTL) JSCC model that maintains good performance on both tasks.
Although these JSCC models achieve excellent results, few models consider constellation design issues in the modulation module of communication systems. The output values of the JSCC encoder are continuous, so the full-resolution constellation is complex and dense, which is hard and expensive to implement in radio frequency (RF) systems. Moreover, transmitting the continuous values is also a big burden for the antenna. To quantize the full-resolution constellation into a constellation with finite points, there are some constellation design approaches. In [15], deep learning was applied to the physical layer to transmit the messages, where a fully connected neural network (FCNN) was used to build the transmitter and receiver. There are two output nodes in the last layer of the transmitter, which corresponds to the real and imaginary parts of the constellation. Inspired by [15], a higherorder constellation design model was implemented by FCNN in [16]. Apart from the constellation design of the physical layer, [17] proposed a semantic communication system, and the constellation of coded information is mapped to a finite set by uniform quantization.
To achieve more robust and effective constellation mapping of the full-resolution constellation, two approaches are proposed in this paper. By adding a few learnable parameters to the deep JSCC model and fine-tuning the model, both regular and irregular finite constellations are obtained. To the best of the authors' knowledge, this is the first work to perform learning-based constellation design for deep JSCC. Our main contributions are summarized as follows: 1) We propose a deep JSCC model with constellation mapping, which is suitable for real system implementation. traditional uniform quadrature amplitude modulation (QAM) constellation mapping. The rest of the paper is organized as follows. In Section II, we introduce the system background, including the whole network architecture, constellation visualization and QAM mapping. The proposed constellation mapping methods are introduced in Section III. Experimental results are discussed in Section IV. Finally, the paper is concluded in Section V.
Notations: In constellation mapping, each constellation point p is a complex number, Re{p} and Img{p} denote the real and imaginary part of p, respectively. F ⊂ C denotes the set of full-resolution constellation points, and C ⊂ C denotes the set of finite constellation points. Re{F } and Img{F } ⊂ R denote the set of real and imaginary part of constellation points in F, respectively.

II. BACKGROUND
A. Overall System Architecture The overall system architecture used in this paper is shown in Fig. 1, which comes from our previous work [14]. The whole system consists of an MTL network and an asymmetric AE-based JSCC module. The MTL network performs object detection and semantic segmentation. The JSCC model compresses and transmits the intermediate feature of the MTL network. The wireless channel between the mobile device and the edge server is modeled by an additive white Gaussian noise (AWGN) model. Given input feature z ∈ R B and output feature z ∈ R B , the transfer function of AWGN channel is written as z = z + n, with n ∼ N (0, σ 2 ). The parameter σ 2 is the noise variance, which denotes the channel condition. Besides, to meet the average transmit power constraint of a power normalization layer is put at the end of the encoder. The MTL JSCC model in [14] can support 512× compression for the intermediate feature when trained under AWGN channel. We use the well-trained model to illustrate the proposed constellation mapping methods and verify their effectiveness.
The constellation mapping step is added before the normalization layer in the encoder, which maps the full-resolution constellation to the finite constellation while satisfying power constraint. The constellation mapping methods can also be used in other scenarios, such as STL JSCC.

B. Full-resolution constellation and QAM mapping
We visualize the full-resolution constellation of the JSCC encoder outputs by the visualization method in [15]. Suppose the original dimension of the JSCC encoder outputs is (H, W, C), we reshape it to (H*W*C/2, 2). The last dimension corresponds to the real and imaginary parts of the constellation. From Fig. 2(a), we observe that the points in the fullresolution constellation are very dense, making it difficult to design suitable antennas and RF chains in practice. To facilitate the implementation of the deep JSCC model, it is necessary to quantize the full-resolution constellation into a finite constellation. Furthermore, from the distribution of JSCC encoder outputs in Fig. 2(b), we find that the values are mainly The traditional constellation mapping method is QAM mapping, which applies uniform quantization to the real and imaginary parts of the full-resolution constellation. The constellation points of QAM are finite and regularly distributed. 16QAM mapping is also shown in Fig. 2(a), which applies 2-bits uniform quantization to the real and imaginary parts of the fullresolution constellation. 16QAM divides the full-resolution constellation into 16 rectangular clusters, with points in each cluster mapped to a finite constellation point represented by a red triangle.
However, points in full-resolution constellations may not be evenly distributed in most deep JSCC models as shown in Fig.  2(b), so QAM mapping may not be the optimal quantization approach to maintain the high performance of tasks.

III. METHODLOGY
In this section, full-resolution constellation is divided into clusters, and each cluster contains a set of constellation points. Quantizing the full-resolution constellation to the finite constellation is treated as mapping the full-resolution constellation points in the same cluster to one finite constellation point. We propose two methods in Section III-A and III-B, respectively, to map the full-resolution constellation to regular and irregular finite constellations. In this section, we propose a method to map F to C, by mapping the data in Re{F } and Img{F } to Re{C } and Img{C }, respectively. Real part of the data, Re{F }, is taken as the example to illustrate the approach.
Suppose the range of Re{F } ⊂ R is [v min , v max ], and the mapping result set is Re{C , and the input data in the i th interval is mapped to c * i for i = 1, 2, ..., M. A simple diagram with M = 4 is shown in Fig. 3. The length of each interval indicates the size of the cluster in the real axis direction. In this method, we consider regular constellation points, so Re{C } is set to the same value as the uniform quantization result of Re{F } (i.e.,c * i = v min + (i−1) * (v max −v min )/(M −1)), i = 1, 2, . . . , M ), while D is set to be learnable to adjust the length of the interval.
Similarly, we apply this method on Img{F } to adjust the sizes of the clusters in the imaginary axis direction. C can be obtained by the Cartesian product of Re{C } and Img{C }.

Algorithm 1 Mapping to Regular Constellation(MRC)
The pseudocode is shown in Algorithm 1, which we implement in Tensorflow [18]. The working principle of forward pass is as follows: For an input value x ∈ Re{F }, the closest d k is found, and then two adjacent intervals are determined by d k . If x > d k , which means x ∈ interval k+1 , the mapping result is c * k+1 , otherwise the result is c * k . Since the mapping process is non-differentiable, the soft approximation method is proposed in the backpropagation, which is inspired by [19]. The weighted sums of the elements in D and Re{C } are used as the soft assignment of d k and c * k , respectively, and the weights are derived from softmax based on the distances between inputs and the elements in D.
Step function Heaviside 1 determines whether result is c * k or c * k+1 , and the 'gap(c * k , c * k+1 )' is the distance between c * k and c * k+1 . sigmoid is used to perform the soft 1 Heaviside function value is 1 for positive arguments and 0 for others. approximation of Heaviside. δ is a big positive number to make the Result backward close to Result f orward . By using tf.stopgradient, the forward pass output is Result f orward , and the backpropagation output is Result backward . Since Re{C } and Img{C } are set to the same value as the uniform quantization results of Re{F } and Img{F }, respectively, C is distributed as regularly as QAM. However, as MRC adjusts the sizes of the clusters in the real and imaginary axis direction, the cluster size obtained by MRC is different from that of QAM. In addition, MRC only introduces a learnable set D, which has little effect on the complexity of the model.

B. Mapping to Irregular Constellation (MIC)
In this section, we propose a method to map F to C based on clustering, and the finite constellation points in C are adjustable and can be trained end-to-end. The clusters are obtained by clustering the full-resolution constellation points in F based on the distance to the points in C.
Suppose the learnable finite constellation point set is C = {c 1 , c 2 , ..., c N } ⊂ C, and the initial value of C is set to the same value as QAM. The value of C will be adjusted based on task performance. For an input constellation point p ∈ F, the clustering algorithm based on the Euclidean distance is used to map p to a nearest constellation point in C. As the process of finding the nearest constellation point is non-differentiable, we implement soft assignment in the backpropagation based on the method in Algorithm 1. The pseudocode is shown in Algorithm 2. δ is the same as MRC. C and clusters are obtained by training the model. As the constellation points in F are not evenly distributed, better performance can be obtained using irregularly distributed C and irregularly shaped clusters. In addition, MIC only adds a learnable parameter set C, which has little effect on the model complexity.

A. Dataset and training strategy
In the experiments, we use the same dataset as [14], which is part of the Open Image Dataset [20], to train the JSCC model with the addition of the constellation mapping step. Based on the trained JSCC model, the training strategy of the new model has two steps. The first step is to train the parameters in the constellation mapping part and fix other parts in Fig. 1. The second step is to train the parameters of the decoder and feature parsing part. The reason why we fix the parameters of the feature extraction part and the encoder part is that the output data distribution of JSCC encoder should be constant.
The loss function to train the new model is the same as the JSCC model, where the loss function L M T L has two parts, i.e., the loss function of object detection L det and semantic segmentation L seg : (1) The model is optimized by the Adam algorithm with a minbatch size of 32. The iteration number of the first training step is 20K, and the initial learning rate is 10 −3 and decreases twice by factor 10 at iteration 5K and 15K. The iteration number of the second training step is 50K, and the initial learning rate is 10 −5 and decreases by factor 10 at iteration 35K. In addition, the training SNRs are 5 dB and 10 dB, and δ is set to 20.
For the baseline method, we use QAM which is introduced in Section II-B as the constellation mapping step of the fullresolution constellation. The decoder and feature parsing part are trained according to the training strategy.

B. Constellation mapping visualization
To show the difference between proposed methods and QAM, we train the models with MRC, MIC, and 16QAM according to the training strategy, and visualize the constellation mapping results in Fig. 4. The full-resolution constellation is divided into 16 clusters based on different mapping methods. Points in each cluster have the same color and are mapped to a finite constellation point represented by a red triangle.
In 16QAM, all clusters have the same size, and the finite constellation is regular. MRC adjusts the sizes of the clusters based on task performance, but keeps the mapping results the same value as uniform quantization. The cluster sizes in Fig.  4(a) are different from 16QAM, while the finite constellation is the same as 16QAM. The finite constellation points in MIC are obtained based on task performance, and the clusters are constructed based on clustering. The finite constellation in Fig. 4(b) is irregularly distributed and the sizes of clusters corresponding to each constellation point are different from the first two methods.

C. Constellation mapping performance
After training the models with different constellation mapping methods under different training SNRs (i.e., SNR train ), we evaluate the model performance at different testing SNRs (i.e., SNR test ). As mentioned earlier, our model performs both semantic segmentation and object detection. The quality of predicted segmentation masks is measured with mean intersection over union (mIoU) and the metric for evaluating detection performance is the mean average precision (mAP).
The performance of the models on two tasks is shown in Fig.  5. The solid lines are the performance of the proposed methods against 64QAM, and the dashed lines are the performance of the proposed methods against 16QAM. It is clear that both proposed constellation mapping methods outperform QAM on two tasks under different training SNRs. In particular, the performance of MIC is better than that of the other two methods on both tasks under different test SNRs.
Finally, the impact of the algorithms on the computational complexity of the model is analyzed. The computational complexity of the original MTL JSCC model is 127.5 GFLOPs, while the complexity of MRC and MIC is 0.23 MFLOPs and 0.92 MFLOPs, respectively. Therefore, the added module has little effect on the computational complexity of the model.

V. CONCLUSION
In this paper, two novel constellation mapping methods are proposed to quantize the full-resolution constellation of the deep JSCC model. Both regular and irregular finite constellations are obtained. With constellation mapping, the deep JSCC model becomes suitable for practical hardware implementation. We apply the constellation mapping methods to the trained MTL JSCC model proposed in our previous work. The performance of the proposed constellation mapping methods on two tasks is better than the traditional QAM mapping. Although the experiments are only carried out on MTL JSCC in this paper, the proposed algorithms can be easily extended to other deep JSCC models.