{'fraction_non_alphanumeric': 0.0327017769214301, 'fraction_numerical': 0.0076803682295011775, 'mean_word_length': 4.546830933649993, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Robust Deep Learning-Based Beamforming Design for RIS-assisted Multiuser MISO Communications with Practical Constraints

Reconfigurable intelligent surface (RIS) has become a promising technology to improve wireless communication in recent years. It steers the incident signals to create a favorable propagation environment by controlling the reconfigurable passive elements with less hardware cost and lower power consumption. In this paper, we consider a RIS-aided multiuser multiple-input single-output downlink communication system. We aim to maximize the weighted sum-rate of all users by joint optimizing the active beamforming at the access point and the passive beamforming vector of the RIS elements. Unlike most existing works, we consider the more practical situation with the discrete phase shifts and imperfect channel state information (CSI). Specifically, for the situation that the discrete phase shifts and perfect CSI are considered, we first develop a deep quantization neural network (DQNN) to simultaneously design the active and passive beamforming while most reported works design them alternatively. Then, we propose an improved structure (I-DQNN) based on DQNN to simplify the parameters decision process when the control bits of each RIS element are greater than 1 bit. Finally, we extend the two proposed DQNN-based algorithms to the case that the discrete phase shifts and imperfect CSI are considered simultaneously. Our simulation results show that the two DQNN-based algorithms have better performance than traditional algorithms in the perfect CSI case, and are also more robust in the imperfect CSI case.


I. INTRODUCTION
In recent years, with the explosion of mobile devices in wireless communications, it has become increasingly important to improve communication quality of service (QoS) and communication speed. Therefore, numerous works have been proposed to meet these requirements, such as the relaying [1] and massive multiple-input and multiple-output (MIMO) systems [2].
The former actively enhances the signal strength but needs extra hardware consumption and complex signal processing. The latter uses a large number of antennas to exploit the spatial gain, which can effectively improve the spectrum efficiency. However, implementing massive MIMO is still a very challenging task since the high cost, increased power consumption, and constrained physical size and shape.
Reconfigurable intelligent surface (RIS) is a promising technology that greatly improves coverage, throughput, spectrum efficiency, and energy efficiency of wireless communication [3]- [7]. The RIS does not require active radio frequency (RF) chains or complex signal processing.
Instead, it relies on passive components with simpler hardware and less energy consumption than traditional transceivers and relay systems to reflect signals. These passive elements can be controlled by simple programmable positive-intrinsic-negative (PIN) diodes and a microcontroller [8]. The micro-controller adjusts the phase shift and amplitude of each passive element at the RIS to achieve fine-grained three-dimensional passive beamforming so that the signal power is improved at the desired receiver or reduced at the non-desired receiver [9].
Although the RIS has so many excellent performances and wide application prospects in future wireless communications, it still faces many daunting challenges. One of them is the joint designing of the active beamforming at the AP and the passive beamforming at the RIS in the RIS-aided wireless communication system with practical constraints.
Recently, there are plenty of works that have studied the passive beamforming issues from different aspects and setups. In [4], [10]- [12], the authors considered that each element at the RIS has continuous phase shifts for the passive beamforming. Specifically, [4] solved the joint beamforming problem by decomposing the non-convex coupling objective into active transmit beamforming and passive beamforming problems, respectively. For active transmit beamforming, the maximum ratio transmission (MRT) and the minimum mean squared error (MMSE) criterion were used in the single-user and the multi-user situations. For passive beamforming, the semidefinite relaxation (SDR) method was applied. [10] maximized the weighted sum-rate (WSR) of all users by joint designing the active beamforming and the passive beamforming. [11] proposed an alternating optimization (AO) algorithm to maximize the minimum signal to interference plus noise ratio (SINR), subject to a transmit power constraint and a unit-modulus constraint. [12] proposed two computationally affordable approaches for passive beamforming, capitalizing on alternating maximization, gradient descent search, and sequential fractional programming. [13] considered a continuous phase-shift passive beamforming problem in an orthogonal frequency division multiplexing system and the reflection coefficients are then optimized by a low-complexity algorithm based on the resolved strongest signal path in the time domain.
However, it is impractical to achieve continuous phase shifts due to the hardware limitation.
Therefore, there are many works that have taken into account the discrete phase-shift situation, such as [14]- [17]. Specifically, [14] relaxed the discrete phase-shift constraint to the continuous phase-shift constraint. Then, the final results were obtained by discretizing the continuous phases.
In [15], the non-convex WSR problem with discrete phase shifts was firstly decoupled via Lagrangian dual transform, and then the active and passive beamforming can be optimized alternatingly. [16] formulated the problem as minimizing the transmit power, and optimized the continuous transmit precoding at the AP and the discrete phase shifts at the RIS. [17] proposed a hybrid beamforming scheme and formulated a sum-rate maximization problem with a limited number of discrete phase shifts. An iterative algorithm was designed to solve this problem.
Most existing works assumed that the CSI between AP/BS and RIS and RIS and users is perfect [4], [12], [18]- [23]. However, this assumption is impractical because of the complexity of channel estimation. Furthermore, there have been some works on joint beamforming in the imperfect CSI case. In [10], the joint beamforming problem with imperfect CSI was formulated as maximizing the expectation of the achievable WSR. The sample average approximation method and the stochastic successive convex approximation (SCA) technique were applied to solve this problem. In [24], the authors applied the least square method for channel estimation, explicitly reflecting the channel estimation's error in the SINR expression, which was solved by the SDRbased method with high complexity. In [25], the authors exploited some algorithms such as alternating optimization, a penalty-based approach, SCA, and semidefinite relaxation to design the secure wireless communications assisted by multiple IRSs under the imperfect CSI constraint.
In [26], the secure communication system with self-sustainable IRS was considered under the discrete phase shift and imperfect CSI constraints. The complex optimization objective was solved by decoupling the coupled variables.
To optimize active beamforming and passive beamforming simultaneously with less complexity, deep learning (DL) technologies have been applied for RIS-aided communication systems in recent years. For example, [27] proposed a supervised learning (SL) method with four layers for the passive beamforming design, which reduces the pilot training overhead but requires active RIS elements for channel training, while the authors reduced complexity at the model training stage via unsupervised learning in [28]. Moreover, [29] introduced a deep Q network (DQN) which does not require labels like SL but with longer training time. Note that [29] also needs active RIS elements for channel acquisition like [27]. Some deep deterministic policy gradient based algorithms have been proposed in [30]- [32]. They all have better performance than DQN-based methods. Besides, the DL-based methods are also applied in many more complex scenarios. For example, [33] proposed a DQN with three layers to beat eavesdropping robustly but with high model training complexity in secure beamforming. [34] focused on energy-efficient beamforming with DQN, and the indoor beamforming was considered in [35] via a DL-based method.
Currently, there are many DL-based methods for the optimization of continuous RIS phase shifts with the perfect CSI, but as far as we know, there is no DL-based method for the optimization in a practical situation that the discrete phase shifts and imperfect CSI are considered yet.
Motivate by these, in this paper, we consider the joint beamforming problem with discrete 5 can be applied directly in the stochastic gradient descent-based DQNN to take the errors caused by a quantization into joint optimization, thereby improving the performance of RIS-assisted systems.
• Secondly, we analyze the mismatch problem of the DQNN-based algorithm and solve this problem to obtain optimal parameters of DQNN via a comparative search method. Besides, we also develop an improved DQNN (I-DQNN) by applying a penalty term to lower the overhead of the parameters' optimization for more efficient training in the face of 1-bit quantized discrete phase shift. Simulation results show the efficiency of the proposed I-DQNN.
• Finally, for the discrete phase shifts and the imperfect CSI case, we formulate the joint beamforming problem with an expectation operator as a more straightforward WSR problem, which our proposed DL-based algorithms can readily solve. Simulation results show that the two algorithms outperform traditional methods, albeit their significantly reduced computational complexity.
The rest of this paper is organized as follows. Section II introduces the system model and the problem formulation for the RIS-aided multiuser MISO downlink communication system.
In Sections III, we introduce the structure and application of DQNN and I-DQNN. Section IV presents the simulation results of our proposed algorithms. Finally, we conclude the paper in Section V.

A. System Model
In this paper, we consider a basic RIS-aided multiuser MISO system as illustrated in Fig.   1. The system consists of one AP with M antennas, one micro-controller, K single-antenna users and one RIS. The RIS is equipped with N = N x * N y passive reflecting elements in a rectangular arrangement. N x and N y represent the number of elements in each row and each column, respectively. During the downlink period, the M antennas of the AP transmit K data streams to different K users. We assume that the direct links between the AP and the users are blocked by some obstacles, which is practical in many complex environments like indoor. Note that our work is also adapted to the situation where direct links exist by simply adjusting the loss function and network input.
Let G ∈ C N ×M and h r,k ∈ C N ×1 , k = 1, ..., K denote the channel matrices between the AP and the RIS, and the RIS and the k-th user, respectively. For simplicity, we assume that all the channels experience quasistatic flat-fading and the CSI is known at both AP and RIS [36]. Let θ H ∆ = [v 1 e jϕ 1 , v 2 e jϕ 2 , · · · , v N e jϕ N ] denotes the reflection coefficient vector of the reflecting elements at the RIS where v l and ϕ l are the amplitude and phase shift, respectively. For simplicity, we assume that the RIS can completely reflect the incident signals, which means that the amplitudes of all reflection coefficients are constant and set as v l = 1, l = 1, 2, . . . , N. Thus, the reflection coefficient vector can be rewritten as θ H ∆ = [e jϕ 1 , e jϕ 2 , · · · , e jϕ N ]. In practice, only a finite number of discrete values for phase shifts are available. We assume that b denotes the bits controlling per element and B = 2 b is the discrete phase-shift levels. Then we can uniformly quantize the continuous phase shifts in the range of [0, 2π), and the set of these phase shifts is The signal received at user k in a downlink transmission can be expressed as where Θ = diag(e jϕ 1 , e jϕ 2 , · · · , e jϕ N ) is the phase shift matrix of the RIS, W ∈ C M ×K is the active transmit beamforming at the AP, x ∈ C K×1 is the vector of the transmitted data streams, with zero mean unit variance entries, E{|x| 2 } = 1, and z k ∼ CN (0, σ 2 k ) is the additive white Gaussian noise (AWGN) in the receiver of k-th user. For simplicity, we assume that σ 2 k = σ 2 . The received signal model (1) can be further written as: where w k is the k-th column vector of the active transmit beamforming matrix W . In (2), h H r,k ΘGw k x k is the desired signal at the k-th user and K n,n =k h H r,k ΘGw n x n is treated as cochannel interference.
Then the SINR at the k-th user is given by Besides, the transmit power constraint of AP is

B. Problem Formulation
All works we have done for the joint beamforming problem in this paper are based on maximizing the WSR of all the users, subject to the transmit power and discrete phase shifts constraints. At first, we consider this problem under the assumption that the CSI is perfect at both AP and RIS. Then we extend it to the more practical and complex case that only imperfect CSI is available at both AP and RIS.

1) Perfect CSI:
Although it is impractical to consider perfect CSI, it helps measure the proposed algorithm's fundamental performance and be used as a benchmark method when considering the joint beamforming problem. More importantly, many SL-based methods use beamforming vectors obtained in perfect CSI as labels for training. Therefore, we can also provide labels for SL in this way [37], [38].
We extend the WSR maximization problem proposed in [10] to a new one with the discrete phase shifts constraint, which is given by where f A (W , θ) is the WSR of all users, q k is used to represent the priority of the k-th user and θ n is the n-th reflection coefficient of the RIS. It can be seen that (5) is a non-convex optimization problem due to the non-convex objective function and the constraints. The optimal result can be obtained by the exhaustive search method. However, it is nearly impossible when the number of elements of RIS is very large. Therefore, many existing works try to find the suboptimal solution based on the AO algorithms. Specifically, in each iteration, the AO algorithms optimize the W with fixed θ and then optimize the θ with the fixed W . The difference between them is on the aspects of methods when optimizing W and θ. Except for the max-sum-rate problem considered in this paper, P1 can also be extended to solve the problem having minimum individual user data rate constraints. For example, minimum user data rate constraints can be easily transformed and added to the objective optimization function and solved by the proposed deep learning method.
2) Imperfect CSI: To optimize the passive beamforming vector θ, the channels should be estimated firstly. However, it is difficult to obtain an accurate CSI due to the complexity of channel estimation. Therefore, the imperfect CSI case is more practical. In this paper, we assume that the CSI obtained at both AP and RIS is imperfect. Then, the channel estimation errors h e and G e can be expressed as h e = h r,k −ĥ r,k , where h r,k and G are the true channels,ĥ r,k andĜ are the estimated channels. Note that the specific channel estimation mechanism is not discussed since it is not our paper's focus. We just assume that the MMSE method is applied in the channel estimation, which makes the estimation errors and the estimated channels irrelevant [39], [40].
Then, (2) can be further expressed as Similarly, (3) is rewritten as We assume that the estimation errors h e and G e follow the circularly symmetric complex gaussian (CSCG) distribution. Therefore, the true channels h r,k and G can be modeled as a realization from the sample space F = {h r,k (ξ) , G (ξ) , ∀k, ∀ξ} dominated by the knowledge of the imperfect CSI and the channel estimation errors' distribution, where ξ denotes the index of the random realizations drawn from F .
In this setup, we formulate the optimization problem in the imperfect CSI case as s.t. |θ n (ξ)| = 1, ∀n = 1, 2, · · · , N, The model of P2 is inspired by [10]. The difference is that [10] decomposed the nonconvex optimization problem into two parts: active beamforming with perfect CSI and passive beamforming with imperfect CSI. While P2 considers both imperfect CSI at the BS and RIS. P2 is a stochastic optimization problem with the expectation operator, which is complex to solve because of the complicate probability density function of the sample space F .

III. PROPOSED DEEP QUANTIZATION NEURAL NETWORK FOR PASSIVE BEAMFORMING
To address the above problems, we propose a deep quantization neural network framework, called the DQNN. The DQNN has a two-stage design, offline training and online prediction, which are not the same with the normal DL methods. During the offline training, we train the DQNN with training samples consisting of estimated channels. After the training is completed, we replace the scalar quantization layer with a real quantization layer whose outputs are real phases in the set S during the online prediction. Then, we explore the parameter settings and propose the I-DQNN to adapt to different resolutions of discrete phase shifters. Finally, we also extend DQNN and I-DQNN to the P2. Both DQNN and I-DQNN are unsupervised learning without labels of the optimal RIS reflection vector and transmit precoding matrix.

A. The Structure of DQNN
The structure of DQNN is illustrated in Fig. 2, consisting of a DNN, a normalization layer, a scalar quantization layer, a real quantization layer, and a lambda layer. The inputs of DQNN are the estimated channelsĥ r,k andĜ. Because (10) shows that the active transmit beamforming W and the reflection coefficient vector θ can be obtained by a function ofĥ r,k andĜ.
The universal approximation theorem shows that a feedforward neural network with only a single hidden layer and a finite number of neural units can fit functions of arbitrary complexity with arbitrary precision [41]. Therefore, the DNN can learn the mapping function shown in (10).
Note that the elements of W and θ are all complex. However, the neural network is based on real values. Thus, we do not directly learn the mapping of the estimated channelsĥ r,k andĜ to the complex W and θ. Instead, we learn the vector consisting of the real and imaginary parts of the W and the real phases of the θ, which is different from that many DL-based methods learn the real and imaginary parts of θ as well as. Because the real phaseφ cont is easier to be processed by our proposed scalar quantization layer than the real and imaginary parts of θ. Theφ cont andφ disc are the input and output, respectively, of the scalar quantization layer and the real quantization layer. For the unit-modulus constraint in P1, we use a lambda layer to convert the real-value phaseφ disc into a complex reflection coefficientθ, which has a unit modulus. The function of the lambda layer is given bŷ To implement the power constraint defined in (5), a normalization layer is employed at the outputs Re(Ŵ pred ) and Im(Ŵ pred ) of the DNN. The function of the normalization layer is expressed asŴ Then we useθ,Ŵ to calculate the loss function for P1, which can be expressed as where L is the batch size of the training samples. The loss function represented in (13) is the negative sum of the WSR of a batch, which means that the maximum WSR can be obtained by minimizing the loss function.

B. Scalar Quantization Layer
In this subsection, we introduce the scalar quantization layer inspired by [42]. While considering using DL-based methods to solve P1, one of the most critical challenges is the constraint of discrete phase shifts. The key idea of DQNN to address this challenge is that we first obtain the continuous phasesφ cont by relaxing the discrete phase shifts constraint to the continuous phase shifts constraint. Secondly, we employ a continuous-to-discrete mapping called quantization layer, which can quantize the continuous phases to the discrete phases in set S. To address this challenge, we construct a scalar quantization layer based on a soft-to-hard quantization technique, which is based on approximating the non-differentiable quantization function by a differentiable one [43]. To be precise, we use a sum of shifted hyperbolic tangents as the activation function, which is differentiable, to replace the non-differentiable quantization function. Note that other differentiable activation functions can also be used, such as sigmoid and arctangent. They have similar property as the shifted hyperbolic tangent by controlling their steepness, amplitude, quantization area, and other characteristics.
It can be seen from Fig. 3 that shifted hyperbolic tangents can approximate the quantization function very well, and it is a differentiable function that is easy to be applied in SGD-based DL. To limit the output phases to [0, 2π), we also make a little adjustment to the non-linear function, which can be expressed as where

C. Offline Training and Online Prediction
DQNN consists of two different stages, offline training and online prediction. One novelty in this work is to make the network structures and strategies different in offline training and online prediction. In the offline training, DQNN uses the scalar quantization layer as a continuous-todiscrete approximate mapping. However, the phases obtained in this way are still continuous.
Therefore, we also need to quantize the scalar quantization layer's continuous outputs to discrete phases during the online prediction according to the practical phase shifter's resolution.
To achieve the above requirements, we use a real quantization layer to replace the approximate scalar quantization layer in the online prediction, which can be expressed as where ρ 0 ≤ ρ 1 ≤ · · · ≤ ρ B−1 (when this condition is not satisfied, we re-sort and re-index the parameters according to the size order), and ρ i determines the decision region of each step function in the scalar quantization layer, which is because when tanh (c i (x − ρ i )) = 0, we can get x = ρ i as the decision region.
The function Q R quantizes the continuous phases to the corresponding discrete phases according to the real quantization resolution and the scalar quantization layer parameters learned in the offline training. Note that the continuous function Q A is an approximation of the quantization function Q R . When the Q R is used to replace the Q A , there is a gap between their outputŝ ϕ disc , resulting in a decrease in the corresponding WSR. This gap is controlled by c. It can be seen from Fig. 3 that the steep region A of the quantization function will become smaller as c increases, which indicates that function Q A is getting closer to function Q R . Moreover, it will reduce the gap of WSR, which means that the performance of online prediction will be consistent with the performance of offline training. However, the smaller the region A is, the greater the restriction on the inputs of the scalar quantization layer is. That will increase the difficulty of network training and convergence. On the contrary, when the learning ability of the neural network is sufficient and c is small, a large amount ofφ cont may be distributed in region A, which will cause the gap to become larger. Even if we get the optimal WSR in the training stage, it is also possible to obtain a small WSR in the online prediction because of this gap. Therefore, one of the keys for DQNN to obtain better performance is to choose the optimal parameter c.
To obtain the optimal c, we propose a comparative search method. More specifically, we initialize c randomly at first. Secondly, we use the initial c to train the DQNN with training samples. After the training is completed, the WSR t of the validation samples is obtained. Thirdly, we save this model and predict the reflecting elements' continuous phases of validation samples.
Then we use Q R to quantize the continuous phases to discrete and calculate the corresponding WSR p . We also need to calculate the gap between WSR p and WSR t , which is defined as When the gap G i is less than our preset threshold τ , theφ cont are almost distributed in flat region B, shown in Fig. 3. Then, we need to reduce c for a possible more sufficient training.
This would make better performance with a tradeoff between the gap and the training effect.
Otherwise, increase c for a smaller gap. Then repeat the above process until the WSR p no longer increases. The details of the DQNN-based algorithm are shown in TABLE I.

D. I-DQNN-based Passive Beamforming
In this subsection, we propose an improved network structure, I-DQNN, shown in Fig. 4, based on DQNN. As mentioned above, DQNN relies on c to control the gap between the WSR of offline training and online prediction. However, the process to determine c by the comparative search method is complicate somehow.
Compared with DQNN, we add a gradient-based constraint derived from (14) to I-DQNN to solve the problems as mentioned above. The constraint is defined as f cons is based on the gradient of Q A for x which is represented in Fig. 5. We use two tanh functions to limit the value of c i (x − ρ i ) and −c i (x − ρ i ), respectively, and also to avoid the exponent being too large, which is harmful to train. The detailed derivations are shown in Appendix A.
We construct the loss function of I-DQNN as which consists of the negative of weighted sum rate and the penalty term λ·f cons . During training, our goal is to minimize the loss function, which is equivalent to maximizing the weighted sum rate and minimizing the penalty term. Maximizing the weighted sum rate is our original optimization objective, and minimizing the penalty term aims to eliminate the gap between offline training and online prediction. In I-DQNN, theφ cont are used as the input x of f cons .
The smaller f cons means that moreφ cont fall into the quantization function's flat region B. There will be no big gap problem when online prediction uses a real quantization layer to replace the scalar quantization layer.
Conversely, when f cons is bigger, moreφ cont are in the steep region A of the quantization function, resulting in a more severe gap. Therefore, we add f cons to the loss function so that ϕ cont can be limited and distributed in region B as much as possible. λ is the coefficient of the penalty term, and its role is to strike the balance of the significance of these two parts in the loss function. In the following, we propose a heuristic method to determine the value of λ. Firstly, we set c = 1 as the initial value. Secondly, this initialized c is applied to the pre-training of I-DQNN to get the corresponding W SR c and f cons,c . Noting that, in this pre-training process, the loss function is not (18) instead (13). Because, our purpose is to find the f cons,c with the optimal W SR c for the determination of λ, which can be calculated by In the DQNN parameter optimization process, we assume that K times comparative searches are required to get the best parameter c. Therefore, we need to repeat K times step 2 and step 3, including K times offline training and K times online predictions of validation samples with the real quantization layer. In the I-DQNN parameter optimization process, only one-time pretraining of I-DQNN is necessary to obtain W SR c and f cons to calculate the optimal parameter λ, which means that one-time offline training and online prediction are required. Therefore, we can find that the parameter optimization process of I-DQNN is much simpler and determined than DQNN. Output:θ,Ŵ .

E. Extend the DL-based Algorithms for The Imperfect CSI Setup
In this subsection, we extend our proposed DL-based algorithms to solve P2 with the imperfect CSI setup.
It can be seen that P2 is a challenging problem due to the expectation operator. In [10], the expectation operator can be solved by the sample average approximation method and stochastic SCA technology, which needs to choose different samples from the space F in each iteration.
However, this method needs many iterations to ensure enough samples are taken from F in the practical online application. Instead, this sampling process can be easily transplanted to the offline training process of our proposed algorithms. Because the distribution of the channel estimation error can be learned from the large amount of channel training data collected, which does not cause extra complexity in the practical online application.
More specifically, we generate the input training data set with the estimated CSI at first. Then, we obtain enough samples from the corresponding F to build the average loss functions of the same form, where J is the number of the samples obtain from F , i = 1 is for DQNN, i = 2 is for I-DQNN, Loss i,j represents the original loss function of the j-th sample from F .
The offline training process of the imperfect CSI case is the same as that of the perfect case, except that the loss function is replaced by (20). After the offline training is completed, the online prediction process is the same as the perfect CSI case without a complex sampling process.

IV. SIMULATION RESULTS
This section presents the simulation results to demonstrate the effectiveness of the proposed DL-based algorithms in the joint beamforming task of the RIS-assisted multiuser MISO system with discrete phase shifts and imperfect CSI. The details of the system refer to [10] and are illustrated in Fig. 6, in which one AP equipped with 4 antennas locates at (0, 0), and 2 users with a single antenna randomly distributed in a circle centered at (50, 10) with radius 2 m under the Cartesian coordinate system in meter (m). We assume that the RIS with a uniform rectangular array (URA) is located at (50, 0). We set threshold τ = 0.005.
We consider the large-scale fading and the small-scale fading to describe the real channel environment. Specifically, the path loss of large-scale fading can be expressed as β (d) = β 0 where d represents the distance between the individual links, β 0 = −35.6 dB is the corresponding channel gain when the reference distance d 0 = 1 m, and p represents the path loss exponent.
We set p = 2.2 for the channel AP-RIS and RIS-user. For small-scale fading, we use the Rician channel model for two individual links, expressed as where κ G and κ r,k are the Rician factor of AP-RIS link and RIS-user link, respectively. We set CSI samples are the estimated channels with errors compared to the real channels. The error can be expressed as where x is one element of the channels, andx is the corresponding estimate value. We assume that the estimate error x −x follows zero mean complex Gaussian distribution, and all these elements have the same normalized MSE η. Noting that, η = 0 for the case of perfect CSI. We

A. Benchmarks for Comparison
We compare the performance of the proposed DL-based algorithms with the following benchmark schemes for the cases of perfect CSI and imperfect CSI: • Scheme 1 (AO algorithm): The algorithm proposed in [24] for the joint beamforming with discrete phase shift constraint based on the AO and the successive refinement algorithm.
• Scheme 2 (Extended BCD): For the perfect CSI case, the continuous phase shifts of passive beamforming are firstly obtained by the BCD-based algorithm [10] in each iteration. Then we applied the successive refinement algorithm proposed in [24] to obtain discrete phase shifts from the optimized continuous phase shifts. For the imperfect CSI case, the continuous phase shifts of passive beamforming are solved by the imperfect CSI version of BCD-based algorithm proposed in [10], and the other process is the same as the perfect CSI's.
• Scheme 3 (Random discrete phase shifts): θ is initialized by randomly choosing value in set S, and then W is optimized by WMMSE for both perfect CSI and imperfect CSI.  with different N, the optimal c of DQNN obtained by comparative search method and the λ calculated by (19) are illustrated at TABLE III and TABLE IV respectively. As we can see that

D. Performance Versus η
In this subsection, we discuss on the imperfect CSI case which is more complicated to solve in traditional algorithm but simpler in our proposed DL-based algorithms. In Fig. 9, we compare the performance of different algorithms to solve P2 versus P t with the fixed N = 50. We observe that the performance of all algorithms increases with the increase of P t under different η. The difference is that in the BCD-based algorithm, as P t gradually increases, the performance loss between perfect CSI (η = 0) and imperfect CSI (η = 0.6) gradually increases. For example, when b = 1 bit and P t = 6 dBm, the loss is 5 dB, and when P t increases to 10 dBm, the loss increases to 8 dB. On the contrary, the performance loss of our proposed DL-based algorithms between perfect CSI (η = 0) and imperfect CSI (η = 0.6) remains almost unchanged with the increase of P t, and it is about 3 dB when b = 1 bit and b = 2 bits.
Next, in Fig. 10 our proposed DL-based algorithms have better and more stable performance than BCD-based algorithm.
Finally, in Fig. 11, we fix N = 50 and P t = 5 dBm to study the performance of various algorithms with different error MSE η. It is seen that the performance of all algorithms decreases with the increase of η. We compare the DL-based algorithms and the BCD-based algorithm with the same b. We can see that the performance of the former is much better than the latter and the gap between them increases as the η increases.  BCD-based algorithms.

E. Comparison Of The Parameters Optimization
In the above experimental results, we can find that DQNN and I-DQNN have very similar and excellent performance with the premise that they all have the optimal parameters. However, the complexity of determining their optimal training parameters is different. As mentioned above, to get the best results in DQNN, we need to find the optimal c, which maximizes the WSR by the time-consuming comparative search method. In contrast, for I-DQNN, the parameter optimization process is much simpler. TABLE III illustrates the optimal c of DQNN. We can observe that c can be easily set as 1 for different N when b = 1 bit, since it is easy to train with the lower resolution of discrete phase shifters. However, when b = 2 bits, the training of DQNN is more complicated. Thus the optimal c needs to be determined by the comparative search method.

F. Computational Complexity and Convergence
In addition to measure the performance of algorithms for solving the non-convex problem, these algorithms' computational complexity also needs to be considered. In [4], the authors apply the SDR-based algorithm to the passive beamforming with the fixed active transmit beamforming of MRT. Although it achieves good performance, the complexity of the SDR technique is O (N 6 ), which is very high with a large number of the elements at the RIS. In [10], the total complexity of the proposed BCD-based algorithm is O (I O (2KNM + KM 2 + K 2 N 2 )), where I O is the number of iterations. Paper [17] shows that the proposed iterative algorithm for the joint beamforming with discrete phase shifts has a complexity O 2 b N 2 . Although we need much time for the offline training of DQNN, the complexity of online prediction is small. For simplicity, we ignore the calculation of the BN layers and consider the flops of the FC layers as the computational complexity, which is 32H * 16H + 16H * 8H + 8H * 4H + 4H * H = 676H 2 , where H = N + 2KM. Therefore, the total complexity is O (676(N 2 + 4KMN + 4K 2 M 2 )). It can be concluded that the DQNN-based algorithms have a comparable complexity to the existing state-of-the-art algorithms. Fig. 13(a) and Fig. 13(b) demonstrate the convergence trends of DQNN and I-DQNN, respectively. We can see that DQNN and I-DQNN converge within 100 epochs, and the training time is short. At the same time, when using the early stopping strategy, the training time would be further shortened.

V. CONCLUSION
In this paper, we investigated the RIS-aided multiuser MISO downlink communication system with discrete phase shifts and imperfect CSI. Firstly, we proposed a DL-based algorithm, DQNN, to solve the joint active transmit beamforming and passive beamforming at the RIS with discrete phase shifts in the perfect CSI case. For the constraint of discrete phase shifts, the DQNN applies a scalar quantization layer to approximate the real quantization function which cannot be achieved in a SGD-based network in offline training, and a real quantization layer to map the continuous phases into the discrete phases during the online prediction. Secondly, we proposed the improved DQNN, I-DQNN, which has a simpler parameter determining process for the more complicated training with the higher bits of discrete phase shifters.  (17) We can further rewrite formula (14) as Let u i = e c i (x−ρ i ) and v i = e c i (x−ρ i ) + e −c i (x−ρ i ) , we have We define t i = u i v i and obtain the derivative of Q A (x)