{'fraction_non_alphanumeric': 0.0323498044792037, 'fraction_numerical': 0.009931567721293992, 'mean_word_length': 4.488902439024391, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Site-specific online compressive beam codebook learning in mmWave vehicular communication

Millimeter wave (mmWave) communication is one viable solution to support Gbps sensor data sharing in vehicular networks. The use of large antenna arrays at mmWave and high mobility in vehicular communication make it challenging to design fast beam alignment solutions. In this paper, we propose a novel framework that learns the channel angle-of-departure (AoD) statistics at a base station (BS) and uses this information to efficiently acquire channel measurements. Our framework integrates online learning for compressive sensing (CS) codebook learning and the optimized codebook is used for CS-based beam alignment. We formulate a CS matrix optimization problem based on the AoD statistics available at the BS. Furthermore, based on the CS channel measurements, we develop techniques to update and learn such channel AoD statistics at the BS. We use the upper confidence bound (UCB) algorithm to learn the AoD statistics and the CS matrix. Numerical results show that the CS matrix in the proposed framework provides faster beam alignment than standard CS matrix designs. Simulation results indicate that the proposed beam training technique can reduce overhead by 80% compared to exhaustive beam search, and 70% compared to standard CS solutions that do not exploit any AoD statistics.


I. INTRODUCTION
With the availability of a large amount of bandwidth, mmWave communication can support massive sensor data sharing in vehicular networks [1]- [3]. Due to the large antenna arrays de- Yuyang  ployed at mmWave and the high mobility in vehicular settings, fast and efficient beam alignment strategies have to be designed [4]. For example, exhaustive search-based beam alignment where a radio scans through all the beams from a pre-defined codebook can result in substantial training overhead. The exhaustive search-based solution may not be suitable for mmWave vehicular communication due to a large number of candidates in typical beam codebooks. The mobile nature of terminals in a vehicular setting can impose a strict timing constraint for the beam alignment problem. As a result, exhaustive search-based beam alignment may not meet requirements like low latency and high reliability expected in mmWave vehicular networks [5], [6].
Fast beam alignment in mmWave communication can be achieved through compressed sensing (CS) or side information-aided beam search. CS-based beam alignment solutions exploit sparsity of mmWave channels and require less training overhead than exhaustive search [7], [8]. The performance of CS-based solutions depends on the choice of the CS matrix used to obtain channel measurements. Side information, ubiquitous in vehicular communication, can be exploited for efficient mmWave channel estimation or beam alignment [8]- [14]. Different types of side information can be obtained from sensors mounted on vehicles, including radar, LiDAR and GPS [15]- [19]. Side information can be shared among vehicles by different wireless protocols such as dedicated short-range communication (DSRC), LTE or mmWave communication [1], [20], [21]. Situational awareness of vehicles can also be leveraged to extract environment information and further narrow down the beam search space and reduce beam training overhead [11], [12], [22]- [24]. One limitation of the side information-aided beam selection solutions is that they rely on fully-connected vehicles to share sensor data and enable sensor fusion. Second, none of the approaches leverage the spatial structures associated with vehicular channels for beam selection.
Vehicles channels are temporally correlated, which makes online learning a potential candidate for link configuration in vehicular settings [25]- [27]. Using online learning, a radio can dynamically adapt to the new channel statistics. In [28], beam selection was performed using vehicle arrival directions. In [24] and [29], receiver location was used for online beam selection in mmWave vehicular communication. Such online learning-based beam alignment solutions, however, do not exploit the sparsity of mmWave channels. Algorithms that integrate online learning-based solutions with sparsity-aware solutions may further reduce the training overhead.
In this paper, we assume a vehicular communication setting where a BS serves vehicles in its coverage area. We propose an online beam alignment framework that leverages channel sparsity and the angle domain statistics of the channel. Using our framework, the BS learns the AoD distribution of the paths corresponding to the vehicles in its coverage area. The AoD distribution learned at the BS is further used to optimize the CS matrix. It is important to note that the proposed approach only uses CS-based channel measurements acquired at mmWave, which are used for beam alignment. As a result, our framework does not require fullyconnected vehicles, frequent sensor or out-of-band information sharing [8], [24], [29]. We assume that AoD distribution learning and CS-based computations are implemented at the centralized infrastructure, i.e., the BS. The main contributions of our work can be summarized as follows. • We propose a novel solution for mmWave beam alignment leveraging an online learningbased CS matrix design. We develop a 2D-convolutional CS (2D-CCS) technique that optimize the CS matrix based on the AoD distribution learned at the BS. According to the CS matrix design, the BS applies beam training vectors for the users to acquire compressive channel measurements. Subsequently, the BS updates the AoD statistics based on the channel feedback received from the users. The optimized CS matrix can be obtained when the estimation of AoD distribution converges. • We show that online CS matrix learning using our framework is analogous to a MAB problem [30]. We use the UCB algorithm for online CS matrix learning and investigate different exploration-exploitation methods to estimate the AoD distribution [31]. Furthermore, we propose solutions to calculate the CS matrix using a variation of the estimated AoD distribution with a reduced number of measurements. We apply different exploration terms to UCB based on the levels of confidence in the AoD distribution learning.
• We validate our proposed solution with comprehensive simulations. We demonstrate superior performance of the proposed solution over exhaustive beam search and conventional CS using the exact AoD prior. We show that the proposed online learning-based AoD distribution estimation technique converges to the ground truth using a statistical distance metric. Last, we evaluate the effectiveness of the proposed exploration-exploitation methods.
We would like to highlight that our online learning-based solution is different from the common adaptive CS setting in which the rows of a CS matrix are modified during signal acquisition [32].
Such an approach requires continuous feedback and may result in a substantial training overhead.
In our framework, the CS matrix at the BS remains constant during the beam alignment process of any receiver, and is updated across different channel instances.
The rest of the paper is organized as follows. The problem is motivated in Section II, which demonstrates a specific structure in the AoD distribution at the BS that can be leveraged for datadriven CS in mmWave vehicular communication. The system model is explained in Section II-A.
The channel model is provided in Section II-B. A new CS solution that is well-suited to the AoD prior is proposed in Section III. The online CS matrix learning problem is introduced in Section IV. The application of UCB for the online sensing matrix learning is shown in Section IV-A and Section IV-B. The exploration-exploitation tradeoff is explained in Section IV-C. Comprehensive numerical results are demonstrated in Section V. The final conclusions are drawn in Section VI.
Notation: A is a matrix, a is a column vector and a, A denote scalars. Using this notation A T , A and A * represent the transpose, conjugate and conjugate transpose of A. The scalar a m or a[m] denotes the m th element of a. The 2 norm of a is denoted by a 2 . The scalar A (k, ) denotes the entry of A in the k th row and th column. The matrix |A| contains the element-wise magnitude of A, i.e., |A| k, = |A k, |. The 1 norm of A is denoted by A 1 . The Frobenius norm is denoted by A F . The inner product of two matrices A and B is defined . Cross product of two sets is denoted as ×.

II. MOTIVATION AND DATASET ESTABLISHMENT
In this section, we first explain the motivation of the work. We then describe our ray tracing simulation setup developed to collect vehicular channel data. Finally, we describe how the channels are computed from the ray tracing data to generate the channel dataset.
Channel statistics in vehicular communication follows regular patterns that are dependent on the site-specific environments [34]- [36]. We consider a scenario where a BS is deployed at road side to provide data services to passing-by vehicles that are equipped with mmWave communication devices. In most cellular scenarios, users can be located anywhere in the whole coverage area. For vehicular communication, however, vehicles travel on predetermined lanes that occupy only a small portion of the entire coverage area. In this case, the distribution of vehicle locations follows a statistical pattern that is related to the road layouts. Such statistical pattern of vehicle location distribution provides some informative and useful prior on the AoD corresponding to BS-vehicle links.
The availability of an informative AoD prior in mmWave vehicular communication settings can be helpful for efficient beam alignment or channel estimation algorithms design. For example, weighted sparse recovery techniques that exploit such a prior can be used instead of standard CS algorithms for mmWave link configuration [8]. Furthermore, the beam training vectors used to acquire channel measurements can be optimized to increase the probability of successful alignment. Determining the AoD prior in vehicular communication scenarios, however, can be challenging. A ray tracing-based approach to compute the AoD prior can be prohibitive in practical settings, as it may require accurate models for the surfaces of all the reflectors.
The framework proposed in this paper learns the AoD prior online using compressed channel measurements, and can potentially adapt to new environments.

A. Simulation setup
We use Wireless Insite, a commercial ray tracing simulator, to establish the channel dataset for evaluation [37]. Ray tracing simulation projects rays from the BS to the physical environment and calculates the channel path information, including path gain, angle-of-arrival (AoA) and AoD between the BS and the vehicles. We consider a street layout with two straight lanes in an urban canyon. In the simulation, buildings are modeled as cuboids with a concrete exterior, located on both road sides with different dimensions. The simulation includes vehicles of different types (large vehicles like trucks and small vehicles such as sedans) randomly dropped on the two lanes. The complexity of ray tracing simulation scales exponentially with the number of surfaces. Hence, all the vehicles are modeled as cuboids with metal exteriors for simplicity.
Complete details of the ray tracing setup can be found in [12]. Our framework can be applied in any realistic vehicular environment as it learns the underlying AoD distribution from the channel measurements. We assume that the BS is mounted on top of street-side lamp-posts, which are higher than buses and are dotted around in cities. Receivers are placed on the roof of the vehicles.
A particular realization of this simulation setup is shown in in Fig. 1.

B. Channel model
We assume that the BS is equipped with a uniform planar array (UPA) of size N x × N y .
For ease of notation, we assume that the UPA has the same number of antennas along the azimuth and elevation dimensions, i.e., N x = N and N y = N . The derivation, however, is not subject to the square antenna constraint. We consider an analog beamforming architecture at the BS. In such an architecture, every antenna of the UPA is connected to a single radio frequency chain through a phase shifter. The use of phase shifters allows the BS to generate a variety of beams that can be used for initial access or data transmission. As the focus of this work is on the transmit beam alignment problem, we assume a single antenna receiver at the vehicles. Therefore, under the multiple-input single output (MISO) system assumption, the antenna domain channel can be represented as a vector h ∈ C N 2 ×1 . The channel vector h can be reshaped into a matrix H ∈ C N ×N . The rows and the columns of H correspond to the elevation and the azimuth dimensions of the UPA. Note that the (r, c) th entry of H represents the channel coefficient between the (r, c) th antenna at the transmitter and the receiver antenna. The matrix representation of the MISO channel allows us to better explain the ideas underlying our beam training design. We consider both LOS and NLOS channels in our simulations.
We assume that the channel can be modeled using L p paths in the propagation environment.
We denote the channel gain of the -th path as α , the phase as β , the azimuth AoD as φ , and the elevation AoD as θ . We assume a half-wavelength spaced UPA at the BS to define the Assuming the channel is narrowband, the MISO channel between the BS and the receiver can be expressed as It should be noted that we can extend our framework to the wideband case using the frame structure in [38]. The techniques proposed in this paper estimate the N × N channel matrix in (2). This approximation is used to update the AoD distribution, which is further used for efficient beam alignment.
The BS applies different phase shift matrices to its antenna array in M The number of channel measurements required to estimate the N × N matrix H can be reduced by using techniques that exploit available statistical pattern in H.
The channel matrix is approximately sparse in the 2D-DFT dictionary at mmWave carrier frequencies [2] when a UPA is assumed. We define the beamspace channel X ∈ C N ×N as the inverse 2D-DFT of H, i.e., The beamspace contains the effective channel coefficients seen when different 2D-DFT-based beams are used at the BS [39]. Thanks to the limited scattering characteristics at mmWave frequencies, the beamspace X is approximately sparse. The support of the non-zero coefficients in X is related to the AoD of the propagation rays in the channel. The sparsity of X allows the use of CS-based algorithms for fast channel estimation or beam alignment. For the tractability of analysis, we assume that X is one-sparse. The one-sparse prior for X is perhaps simplistic, but allows a tractable design of a CS matrix that is well-suited to the prior on X. In particular, the prior is only updated based on the strongest channel component, i.e., the direction that is the strongest in the 2D-DFT beamspace. As a result, our approach cannot learn all the components of the channel and we leave the extension to the future work. The proposed CS matrix design, however, achieves good performance over standard constructions for practical channels. It is important to note that the beamspace channels in our simulations include leakage effects and are approximately sparse.

III. COMPRESSIVE CODEBOOK DESIGN WITH PERFECT AOD PRIOR
In this section, we assume the availability of a known AoD prior and design an optimization technique that optimizes CS matrix based on the AoD prior using a 2D-CCS framework [38].
This optimized CS matrix can be used for more efficient channel measurements to improve the beam alignment performance. In Section IV, we will propose an online learning framework that learns such AoD prior and optimizes CS matrix accordingly.

A. 2D-convolutional CS: Motivation and Background
Channel measurements in 2D-CCS are the projections of the channel matrix onto 2D circulantly shifted versions of a base matrix P [38]. For M channel measurements with 2D-CCS, the BS applies M distinct 2D-circulant shifts of P to its phased array. The set of the circulant shifts used to acquire the M measurements is denoted as Ω, given by The Second, 2D-CCS can exploit the AoD prior very well to design a proper base matrix suited to site-specific layouts of the streets. As the magnitude of the 2D-DFT of a matrix is invariant to its 2D-circulant shifts [40], the 2D-DFT magnitude of all the phase shift matrices in 2D-CCS is exactly the same. The 2D-DFT magnitude of a phase shift matrix represents its beam pattern sampled at discrete angular locations. A reasonable channel measurement strategy under a AoD prior is to use beam patterns that focus more power along the directions that are more likely to have the strongest components in the channels. Now, we explain the mathematical idea underlying 2D-CCS. We define P FC as the flipped and conjugated version of P, i.e., P FC (k, ) = P(N − k, N − ) ∀k, . To demonstrate how 2D-CCS works, we obtain the 2D circular convolution of the channel and P FC as The k, -th entry G(k, ) is the projection of the channel onto the phase shift matrix obtained by shifting P by k rows and columns. We use P Ω (·) : C N ×N → C |Ω|×1 to represent the projection operator that returns the entries of a matrix at the locations in Ω as a vector. We define the noise vector v ∈ C |Ω|×1 . With this definition, the channel measurements in 2D-CCS can be expressed as Therefore, it can be observed that the channel measurements in 2D-CCS are a subsampling of the convolution between H and P FC .
We use the mask concept in [38] to transform the CS problem corresponding to (7) into a partial 2D-DFT CS problem. The transformation will be used to simplify our CS matrix optimization problem. Based on the DFT property, the inverse 2D-DFT of G is given by [40] which is represented as the Hadamard product of two terms, X and Z. The left side of the Hadamard product in (8) is the channel beamspace. The right side of the Hadamard product is defined as the mask. As P F = 1, it can be observed that Z F = N . We name the Hadamard product of X and Z as the masked beamspace [38]. It can be observed from (8) that We substitute the 2D-DFT representation of (8) in (7), which gives As the masked beamspace X Z is sparse, it can be recovered from its subsampled 2D-DFT in (9) with standard partial 2D-DFT techniques. To guarantee successful recovery of the beamspace X, however, the mask Z should be non-zero at all locations. To satisfy such non-zero constraints, a unimodular mask was used in [38]. Denoting the estimated beamspace asX, it was shown that the use of a unimodular mask minimizes the channel reconstruction error ||X −X|| F [38].
Non-uniform masks can be more efficient to acquire compressive channel measurements when the AoD prior is known at the BS. As the AoD prior in vehicular settings follows a certain statistical pattern according to the street layouts, designing a mask Z that is tailored to the prior can result in better beam alignment. In this paper, we define the AoD prior as a probability distribution of the best beam index within the 2D-DFT beamspace. A good mask is one with a larger magnitude along the directions that are more likely to be optimal. As the squared Frobenius norm of the mask is constrained to be N 2 , the mask design problem is analogous to a power allocation problem. In Section III-B, we investigate the mask design problem for a given AoD prior.

B. Mask design with AoD prior
CS matrices in 2D-CCS are parameterized by P, or the mask Z, and the subsampling set Ω. In this paper, we focus on the optimization over the magnitude of the mask and leave optimization of the subsampling set Ω to future work. To this end, we assume . Under such full sampling assumption, (9) can be rewritten as We defineỹ as an N 2 × 1 vector that is obtained by inverting the 2D-DFT in (10), i.e.,ỹ = Due to the unitary nature of the 2D-DFT,ṽ has the same statistics as v, i.e.,ṽ ∼ N c (0, σ 2 I).
We assume that the one-sparse beamspace vector x is an instance of a random variable x that is supported on a discrete event set F. Under the one-sparse prior assumption, F contains the canonical basis vectors e 1 , e 2 , · · · , e N 2 ∈ C N 2 ×1 . The probability that x = e k , i.e., the k th beamspace component is 1, is p k . The AoD prior is defined by the probabilities {p k } N 2 k=1 that sum to one. The event that x = e k corresponds to the case in which the k-th beam direction is optimal in the 2D-DFT beamspace. Now, we derive the probability of successful beam alignment for a mask z. For such a computation, we first derive the success probability conditioned on x = e k . When the k th beamspace direction is optimal, the entries ofỹ can be expressed as For x = e k , beam alignment is successful when |ỹ k | > |ỹ j |, ∀j = k, i.e., Such an event occurs with a probability j =k P (|ṽ j | < |z k +ṽ k |). Considering the N 2 candidates for x, the probability of successful beam alignment is where (a) is derived based on the assumption that the events in F are independent, and (b) follows from the IID noise assumption.
We express the probability of successful beam alignment as a function of the AoD prior, the mask, and the noise variance σ 2 . Conditioned on x = e k , the random variableṽ z k ∼ N 0, σ 2 |z k | .We use Lemma 1 to simplify (18).
Lemma 1. For x and y as IID complex Gaussian with x, y ∼ N (0, ξ 2 ), we have Proof. For convenience of notation, we first rewrite P (|1 + x| 2 ≥ |y| 2 ) as Since both x and y are complex Gaussian, the distribution of follows a non-centered chi-square distribution of degree 2 with λ = 2ξ 2 [41]. The probability in (19) can be derived as where (a) is derived based on the moment-generating function of non-centered chi-square distribution with degree 2 and λ = 2ξ 2 [41].
We use Lemma 1, to rewrite (18) as The goal of this section is to determine the mask z that maximizes the probability in (23).
We now derive a convex optimization problem that maximizes an approximation of the beam alignment probability. As P F = 1, the mask must satisfy the norm constraint z 2 = N . The mask optimization problem is then Unfortunately, the optimization problem in (24) is non-convex. Based on Jensen's inequality, the log-equivalent form of the objective in (24) can be lower-bounded by Fortunately, the lower bound in (26) (24) is also convex in the squared magnitude vector |z| 2 . The optimal mask |Z| opt that maximizes the lower bound on the probability of successful beam alignment can be obtained by the following optimization problem The problem in (27) can be solved using standard convex optimization by considering as the variables. We can observe that (27) shares similar structure to the classical waterfilling problem, by approximating the exponential function in (27) to its linear approximation. In this paper, we do not make such an approximation. Instead, we use the formulation in (27) to find the mask magnitude that is optimized for an AoD prior. We use the CVXPY package in Python [42] to solve the convex optimization problem in (27).
C. Phase recovery from the optimized mask magnitude From (8), it can be observed that the phase shift matrix P is the inverse 2D-DFT of the mask Z. The optimization problem in (27), however, only provides the magnitude of the mask. It is important to note that the phase of the mask cannot be arbitrary. This is because the base phase shift matrix P, i.e., the 2D-DFT of Z, must satisfy the unit-norm constraint of |P| = 1/N . The problem of finding a phase of a matrix such that its inverse 2D-DFT has a known magnitude has been investigated in optics [43]. The Gerchberg-Saxton algorithm in [43] iteratively retrieves the phase of a pair of signals related by Fourier transform, with the availability of the amplitudes of the two signals. In our problem, the base matrix P and the mask Z form a Fourier transform pair, with known amplitudes, i.e., 1/N and the optimized mask |Z| opt . The BS computes the phase of the mask Z opt and the corresponding base matrix P opt by the Gerchberg-Saxton algorithm [43]. The BS applies M N 2 random circulant shifts of the resultant base matrix P opt for the receiver to acquire 2D-CCS-based channel measurements.

IV. ONLINE SENSING MATRIX LEARNING
In Section III, we explained how to design the sensing matrix in 2D-CCS with perfect AoD prior. Such prior can be obtained from a large number of channel realizations measured over a long time, which is similar to the procedure of offline learning. The establishment of such offline dataset, however, may not be easy. First, to identify the optimal beam index, a significant amount of overhead will be introduced due to the deployment of large antenna arrays and complicated channel statistics. Second, it is not clear how many data samples would be sufficient to estimate  Online learning starts with the absence of any useful prior, i.e., the AoD distribution is uniform.
The BS constructs the sensing matrix corresponding to the uniform AoD prior according to the procedure in Section III. The receiver feedbacks the CS-based channel measurements to the BS.
The BS then recovers the channel beamspace based on the channel feedback and updates the AoD distribution with the estimated optimal beam direction. Fig. 2 provides an illustration of the proposed online codebook learning solution. We use a classical online learning solution in MAB, the UCB algorithm, to update the AoD prior.

A. Multi-armed bandits
A MAB is an online learning approach that learns the policy of actions and makes decisions from the rewards observed so far. The MAB problem can be explained as follows. There is a player with K independent machines, each with a reward distribution that is initially unknown A policy π is defined as the mapping from the history H(t − 1) to the action a(t). When the MAB algorithm becomes fully confident about its learned policy, it selects the machine that has the largest expected reward afterwards.
Online learning algorithms require a careful design of the exploration and exploitation. There exists a tradeoff between: 1) exploring the environment to find profitable actions, and 2) taking the empirically best action as often as possible. The tradeoff arises because only a limited number of machines are played at each time slot. The algorithm may repeat playing the optimal machine it observes so far from the rewards. Hence, the algorithm has to explore the environment to improve its knowledge about the reward generating process. In UCB algorithms, such exploration-exploitation status is quantified by upper confidence index associated with each machine [30]. In [44], a family of policies was introduced where the index can be expressed as simple function of the sequence of the actions and rewards of each machine.
In UCB1 algorithm [44], for example, the upper confidence index is the sum of two terms: 1) the empirical mean of the rewards obtained from the sequence of observations, and 2) the width of the confidence bounds that reflects the uncertainty of the player's knowledge. In the basic setting, we assume a K-armed bandit problem [30]. We denote the time that arm k has been selected till time t−1 as T k (t−1). The time slots in which arm k is played are τ k 1 , τ k 2 , · · · , τ k T k (t−1) . The empirical mean of the reward of k-th arm at time t isμ k (t − 1) = 1 . For UCB1 algorithm, the upper confidence index UCB k of the k-th machine at time t is defined as Empirical mean In each time step, the player plays the arm with the largest UCB value. The first term quantifies the knowledge the player accumulates so far as the empirical mean. The second term evaluates exploration as the square root of the inverse of the time the arm has been played. The algorithm suppresses the exploration for arms that have been tested many times, while encourages those arms that have been tried for only few times. The parameter δ can be used to control the extent of the exploration [45]. When time t → ∞, T k (t) → ∞, ∀k and the exploration term in

B. Online sensing matrix learning with UCB
The UCB algorithm fits the online beam alignment problem using a beam scanning-based approach, but its application to the online sensing matrix learning problem is not straightforward.
For exhaustive beam search, there exists a finite-sized beam codebook, where the beam codewords can be treated as machines [11]. The BS can specify which beam direction it wants to measure by simply applying the beam codeword corresponding to the desired beam direction. In our CSbased framework, the BS obtains an approximation of the whole beamspace using compressed channel measurements. Consider an example where the BS wants to test the k-th beam direction.
Instead of obtaining feedback of the beamspace x k directly through beam search, the BS recovers an estimate of the whole beamspacex with CS. The mismatch between the estimated beamspace and the true beamspace, i.e.,x − x, could pose great challenges to the exploration-exploitation design in online learning. The ultimate goal of the paper is to learn the sensing matrix that is best suited for a certain vehicular context. Since both the prior and the sensing matrix are stored and applied at the BS, the online codebook learning model introduces negligible signaling overhead and fully leverages the computation resources at the BS cloud.

C. Exploration-exploitation in online sensing matrix learning
The online sensing matrix learning problem requires an elaborate design of the explorationexploitation strategy. Without the availability of any useful prior, the online learning framework starts with a uniform mask. If the exploration term in (28) is not carefully designed, the BS may apply a uniform mask that does not fully leverage the empirical AoD distribution it learns so far. The use of a uniform mask can lead to poor channel estimation performance and slow convergence of the AoD prior to its real distribution. If the BS uses an AoD distribution that is estimated without sufficient exploration, it can end up using masks that may not have good magnitudes along unexplored directions that are likely to be optimal. The estimated AoD distribution could be more and more biased towards the beam directions that are estimated as optimal in the beginning of the learning. We propose three approaches to achieve the explorationexploitation tradeoff. The first approach targets a smooth update of the AoD prior estimation and a mild exploration over different beams by adding an exploration term to the estimated AoD prior. The second approach, instead of imposing an exploration term to AoD estimation, requires the BS to directly apply such exploration term over the mask. Finally, we demonstrate that by changing the number of CS measurements over time, the proposed approach can achieve a more accurate AoD distribution estimation, especially in the beginning of online sensing matrix learning. The slow decrease of the UCB uniformity guarantees the smoothness of the AoD prior at certain locations during initial exploration. The proposed algorithm is explained in Algorithm 1.

1) UCB with exploration:
2) Mask regularization: The second approach applies exploration directly on the mask Z.
In the first approach, we add an exploration term to the estimated AoD distribution in (28) to minimize the error in the AoD prior estimate that can result in a biased mask. In 2D-CCS, the magnitude of the mask represents the power allocated for sensing along different beam directions that are defined by the 2D-DFT dictionary. We propose to directly apply regularization over the mask. The BS calculates a mask directly based on the empirical AoD distribution, added by an extra exploration term in (28) as in Algorithm 1. The algorithm is provided in Algorithm 2.
It can be observed that Algorithm 1 and 2 share very similar inputs, outputs, and procedures.
The difference is where the exploration (regularization) term is imposed on. The regularization is inexplicitly applied for the AoD estimation of UCB in Section IV-C1, while it is directly incorporated in the mask for probability estimation. Output: Estimated AoD priorp = [p 1 ,p 2 , · · · ,p N 2 ].
, ∀i, and calculate the sensing matrix based on Section III-A -III-C.

4:
Perform 2D-CCS and return the estimated optimal beam direction index s.

5:
Normalize the mask amplitude |z|. Apply Gerchberg-Saxton Algorithm as in Section III-C to obtain the phase of the mask. 6: Perform 2D-CCS with z and return the estimated optimal beam direction index s.

Ts+1
. 8: Update the AoD distribution as the empirical mean p i =μ i (t), ∀i.

9:
Update time of being played for arm i, T i (t + 1) = T i (t) + 1, ∀i.
Linear decrease of M (t) is a simple way of adjusting measurements temporally. For this approach, multiple hyper-parameters, such as ∆ t , ∆ M , M 0 , need to be tested and evaluated.

V. NUMERICAL RESULTS
In this section, we evaluate the proposed online 2D-CCS matrix learning framework with comprehensive numerical results. First, we examine the baseline model with perfect AoD prior, i.e., offline learning. Second, we compare performance of the online sensing matrix learning using different exploration-exploitation techniques proposed in Section IV-C. Last, we demonstrate the convergence of the estimated AoD prior with the proposed online sensing matrix learning algorithms.

A. Offline learning with perfect AoD prior
In Fig. 3, we evaluate the beam alignment performance of 2D-CCS using perfect AoD prior.
The performance metric is the average beam reference signal received power (RSRP). We consider a 32 × 32 UPA at the BS. In the simulation, the transmit power at the BS is adjusted so that the average received SNR for a quasi-omnidirectional transmission is 10 dB. Such an SNR was achieved by using a Golay code to acquire channel measurements [38]. Let P BF represent the beamforming matrix applied to the phased array at the BS. The beamforming gain achieved with P BF is defined as For the perfect CSI case with optimal beamforming, the beamforming matrix at the phased array is P BF, ideal = e jphase(H) /N . We useĤ to denote the channel estimate obtained with our online learning-based 2D-CCS algorithm. The orthogonal matching pursuit (OMP) algorithm [46] was used to solve the sparse recovery problem in our method. The beamforming matrix with the proposed approach is P BF, 2D-CCS = e jphase(Ĥ) /N . For exhaustive beam search, the BS searches through the N 2 beams from a 2D-DFT beam codebook and identifies the optimal beam direction with the largest beam RSRP. For the beam sweeping among the partial beamspace, with the availability of AoD distribution ordered as p = [p r 1 , p r 2 , · · · , p r N 2 : p r 1 > p r 2 > · · · > p r N 2 ], the BS measures the RSRP of the top M beams and selects the one with the largest RSRP correspondingly.   in Section III for the AoD prior in Fig. 4 ii) The mask is also able to cover less important beam directions, e.g., the beamspace on the two sides of the strips along the azimuth direction in Fig. 4.

B. Online sensing matrix learning
In this section, we evaluate and compare the effect of "exploration" on the online learning performance. We consider the case without any exploration, i.e., directly using the mask cal-culated by the estimated AoD distribution for sensing, and the three approaches provided in Section IV-C. Fig. 6 compares the mask learned without any exploration, using a fixed number of measurements. Without exploration, the procedure is identical to Algorithm 1 where δ = ∞ and the exploration term is zero. In this case, the sensing matrix is calculated directly based on the empirical AoD distribution without any regularization. As discussed in Section IV-C, no exploration leads to error propagation and highly biased estimation, which are reflected in the results in Fig. 6. Compared to the mask calculated from perfect AoD prior, the mask without exploration displays some inconsistency in the AoD distribution estimation, and fails to identify several important beam directions. The combined effect of measurement inaccuracy and error propagation due to biased AoD estimation gives rise to a mask that results in poor performance with CS. 2) Exploration and regularization: Fig. 7 and Fig. 8 illustrate the average beamforming loss with UCB in Section IV-C1 and mask regularization in Section IV-C2, compared to the ideal mask calculated from perfect AoD prior in Fig. 5 using different number of measurements.

1) No exploration:
Specifically, we evaluate the performance of our approach for different values of δ. Fig. 8 shows that the gap between the beamforming gain of the mask with perfect AoD prior and the one learned online using a fixed number of measurements is marginal, if the regularization parameter δ is selected carefully. We observe that a low beamforming loss is achieved when 2 log(1/δ) is around 0.1. When 2 log(1/δ) is too large, e.g. 2 log(1/δ) = 1, the mask changes too slowly to capture the information provided by AoD estimation. When 2 log(1/δ) becomes too small, the learning could be too aggressive in the AoD distribution estimation, since it results in a poor mask for 2D-CCS. Similar results of UCB exploration can be observed in Fig. 7. Table I provides a more explicit comparison among the beamforming loss of the mask with perfect AoD prior, uniform mask, the mask achieved with UCB exploration, and the mask with mask regularization, with respect to optimal beamforming using perfect CSI. The beamforming loss decreases with a larger number of CS measurements per channel realization. For M = 40, 50, 60, 80, 100, the optimal beamforming loss with respect to mask with perfect AoD prior is approximately 1 dB. When M = 25, the regularized mask has superior performance to that of the uniform mask. Also, using only 25 measurements throughout the whole learning process, the BS is still able to learn the AoD distribution and results in 2.19 dB beamforming loss compared to the mask calculated using perfect AoD prior. Similar observations can be obtained for different number of CS measurements.
We measure the AoD estimation accuracy based on statistical distance between the estimated AoD distribution and the real AoD distribution of the data samples [47]. We use a statistical distance measure -Hellinger distance [48]. For two discrete distributions P = (p 1 , p 2 , · · · , p K ) and Q = (q 1 , q 2 , · · · , q K ), the Hellinger distance H(P Q) is defined as The Hellinger distance is directly related to the Euclidean norm of the difference of the square root of the probability distribution probability mass functions (PMF). Fig. 9 characterizes the temporal evolution of the AoD estimation with mask regularization using a fixed number of measurements. We compare the Hellinger distance between the empirical AoD distribution and the real AoD distribution, with mask regularization, and that with perfect CSI available at the BS. The real AoD prior is calculated across all 3000 channel realizations.
The empirical AoD prior is calculated at different time steps, e.g., the AoD prior of the first 200, 300 samples, etc. For the perfect CSI case, we assume the BS can always recover the optimal beam direction. The perfect CSI curve provides a baseline reference to evaluate the AoD estimation accuracy during online learning. In Fig. 9, the Hellinger distance decreases sharply in the beginning. When time T < 500, the Hellinger distance with M = 100, 50 closely matches that with perfect CSI. After time T > 1000, the Hellinger distance starts to change slowly. It is also shown that the Hellinger distances of M = 100 and M = 50 are similar.  When M = 20, online learning cannot accurately estimate the exact optimal beam direction, which leads to a large Hellinger distance compared to the perfect CSI case. The performance differences among M = 100 and M = 50 and using the mask calculated with perfect AoD prior are insignificant, as exhibited in Fig. 3 and Table I. The observation implies that the proposed AoD prior-based sensing matrix design is robust to small AoD distribution estimation error, which is favorable to real field implementation that generally has more complicated street layouts and channel characteristics.
3) Updating the number of CS measurements: The online sensing matrix learning is built on the assumption that the CS measurements can roughly give accurate channel estimation, especially in the beginning of the learning. Starting with a large number of measurements when the BS has no prior is a good way to avoid error propagation in the AoD distribution estimation. After measuring more channel realizations, the BS becomes confident about the AoD distribution estimate. In such a case, decreasing the number of CS measurements can result in a reduced training overhead. With mask regularization, the BS further decreases the number of CS measurements in our online learning-based framework. We present the average beamforming loss with a decreasing number of measurements in the 4 -7 rows in Table I. We use M = M min , i.e., the stabilized number of measurements the BS uses for each channel realization. In our evaluation, we fix ∆ M = 20. We update ∆ t in (29) to modify how fast M decreases. The  Fig. 9. Evaluation of the Hellinger distance defined in (31) between the real AoD distribution of in total of 3000 channel realizations, and the empirical AoD distribution learned at different time steps during online sensing matrix learning using mask regularization in Section IV-C2. The regularization parameter is set as δ = 10, which is able to achieve a good explorationexploitation tradeoff as shown in Fig. 7 and  Table   I achieves similar performance compared to the cases with a larger number of measurements M 0 > 200 or ∆ t > 100.

C. Mask convergence
Finally, Fig. 10 compares the mask convergence of 1) UCB exploration in Section IV-C1, 2) decreasing the number of CS measurements combined with mask regularization in Section IV-C3, and 3) mask regularization only in Section IV-C, after 4000 time steps. We can observe that all of the three masks can capture the basic structure and the dominant directions of the AoD distribution in the beamspace. For UCB exploration in Fig 10.a, the mask fails to capture some less likely beam directions. The mask power allocation is more centered around several dominant directions, which makes it difficult to recover a beam direction that has not appeared too often as optimal. The mask convergence for linear measurement number decrease with M 0 = 300, ∆ M = 100 is illustrated in Fig 10.b. As shown in Fig 10.b, the linear measurement decrease procedure achieves better exploration compared to UCB exploration in Fig 10.a. In addition, the mask is less noisy compared to the converged regularized mask in Fig 10.c. The number of CS measurements can be gradually reduced with online learning. The prior learned with our online learning framework enables the BS to perform successful beam alignment with fewer channel measurements than standard CS.

VI. CONCLUSION
In this paper, we proposed a site-specific online sensing matrix learning and beam alignment solution in vehicular communication. In a typical vehicular context, the AoD distribution can be structured due to the regularity of the vehicle trajectories and stationary objects such as buildings and roads. Leveraging such AoD distribution statistics in a vehicular scenario, we proposed a novel online compressive sensing framework using a 2D-CCS-based design. We formulated a convex optimization problem that uses the learned AoD prior to design a CS matrix that maximizes the approximate beam alignment probability. We observed superior performance of the AoD prior-aided sensing matrix design compared to either the standard 2D-CCS without AoD prior or exhaustive beam search. Using only 60 channel measurements, there was less than 2 dB gap between the average beam RSRP of the proposed approach and perfect CSI-aided optimal beamforming.
MmWave vehicular communication has specific channel statistics that need to be exploited for wireless system design. For future work, the design of the subsampling set in 2D-CCS that is well-suited to the AoD prior will be investigated. Furthermore, investigating other priors that are based on beam RSRP is an interesting research direction. In addition to the special street layout and specific channel distribution, more information is already embedded in connected vehicles.
With side information such as location-based situational awareness, the number of measurements can further be reduced if the online learning can reveal the AoD distribution conditioned on the observed side information.