{'fraction_non_alphanumeric': 0.03405360509419784, 'fraction_numerical': 0.008200287405571449, 'mean_word_length': 4.789787039157316, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Model-Based Machine Learning for Communications

We present an introduction to model-based machine learning for communication systems. We begin by reviewing existing strategies for combining model-based algorithms and machine learning from a high level perspective, and compare them to the conventional deep learning approach which utilizes established deep neural network (DNN) architectures trained in an end-to-end manner. Then, we focus on symbol detection, which is one of the fundamental tasks of communication receivers. We show how the different strategies of conventional deep architectures, deep unfolding, and DNN-aided hybrid algorithms, can be applied to this problem. The last two approaches constitute a middle ground between purely model-based and solely DNN-based receivers. By focusing on this specific task, we highlight the advantages and drawbacks of each strategy, and present guidelines to facilitate the design of future model-based deep learning systems for communications.

trained, they tend to implement inference at a lower computational burden and delay compared to their analytical model-based counterparts [4].
Although ML has been the focus of significant research attention over the last decade, it has yet to significantly contribute to practical designs in one of the most important technologies of the modern era -digital communication. The fact that ML-based algorithms, which have revolutionized the fields of computer vision and natural language processing, do not yet play a fundamental role in the design of physical layer communication systems, and particularly digital receivers, may be due to one or more of the following reasons: 1. The large amount of possible outputs impose a major challenge in efficiently applying ML algorithms. In particular, the constellation size of the modulation and the blocklength of the channel code, combined with the time-varying nature of communication channels, leads to an exponentially large number of possible channel outputs that an ML receiver algorithm must be trained on. 2. Traditional deep learning techniques require high computational resources, while communication devices, such as wearable devices and mobile phones, are typically limited in hardware and power. 3. To date, conventional communication schemes, which assume a simplified channel model with parameters that are dynamically estimated, have been very successful.
The third reason is likely to become less relevant as the spectrum congestion of existing cellular standards forces future communication systems to explore new frequency ranges and share spectrum with other application such as radar [5]. As these new frequency bands and spectrum sharing techniques become widespread, the simplified channel, interference, and noise models used in current communication receiver techniques may no longer work well. Moreover, the strict cost, power, and memory constraints imposed on communicating devices lead to the usage of low-resolution analog-to-digital convertors (ADCs) and power amplifiers with dominant non-linearities [6]. This makes the successful application of modelbased techniques significantly more complex. Thus, conventional model-based approaches may no longer be able to meet the performance and throughput demands of future wireless devices, motivating their combination with data-driven approaches based on ML such as deep learning. Such techniques must still overcome the challenges identified above with respect to the large computational resources and data sets needed for training. Despite its unprecedented success, deep learning is subject to several challenges which limit its applicability in some important communication scenarios. In particular, deep neural networks (DNNs) consist of highly-parameterized systems that can represent a broad range of mappings. As such, massive data sets are typically required to learn a desirable mapping, and the computational burden of training and utilizing these networks may constitute a major drawback. For example, consider the two receivers illustrated in Fig. 1.1, which carry out symbol detection using model-based algorithms and model-agnostic DNNs, respectively. The dynamic nature of wireless channels implies that the receivers should track channel variations in order to reliably detect the transmitted messages over long periods of time. To do so, the model-based receiver in Fig. 1.1(a) typically estimates the model parameters imposed on the underlying statistics using periodic pilots. For the same purpose, the DNN-based receiver in Fig. 1.1(b) should periodically re-train its DNN to track channel variations. The fact that doing so requires a large data set leads to a significant decrease of spectral efficiency and increase in computational complexity associated with this training. Furthermore, DNNs are commonly utilized as black-boxes, and thus do not offer the interpretability, flexibility, versatility, and reliability of model-based techniques.
The limitations associated with model-based methods and black-box deep learning systems gave rise to a set of techniques on the interface of traditional model-based communication and ML, attempting to benefit from the best of both worlds [7]. Such model-based ML systems can be divided into two main categories. The first of the two utilizes model-based methods as a form of domain knowledge in designing a DNN architecture, which is then trained and used for inference. The most common example of this strategy is the family of deep unfolded networks [8,9], which design the layers of a DNN to imitate the iterations of a model-based optimization algorithm, and has been utilized in various communication-related tasks [10]. The second strategy, which we call DNN-aided hybrid algorithms, uses model-based methods with integrated DNNs for inference by incorporating ML in a manner that makes the system more ro-bust and model-agnostic. Communication receivers designed using DNN-aided inference include the data-driven implementations of the Viterbi algorithm [11] and the BCJR detector [12].
In this chapter we present an introduction to model-based ML for communication systems. We begin by reviewing existing strategies for combining modelbased algorithms and ML from a high level perspective in Section 1.2, and compare them to the conventional deep learning approach which utilizes established DNN architectures trained in an end-to-end manner. Then, in Section 1.3 we focus on symbol detection, which is one of the fundamental tasks of communication receivers. We show how each strategy, i.e., conventional DNN architectures, deep unfolding, and DNN-aided hybrid algorithms, can be applied to this problem. The last two approaches constitute a middle ground between the purely model-based and the DNN-based receivers illustrated in Fig. 1.1. By focusing on this specific task, we highlight the advantages and drawbacks of each strategy, and present guidelines to facilitate the design of future model-based deep learning systems for communications. We conclude this chapter with a summary provided in Section 1.4.

Model-Based Machine Learning
We begin by reviewing the leading approaches for combining ML, and particularly deep learning, with model-based algorithms. The neural networks in this hybrid-approach are trained in a supervised manner and then used during inference. Then, in Section 1.3 we focus on symbol detection and provide concrete examples on how this approach can be used to design data-driven detectors.
In a broad family of problems, a system is required to map an input variable Y into a prediction of a label variable S. ML systems learn such a mapping from a training set consisting of n t pairs of inputs and their corresponding labels, denoted {s t , y t } nt t=1 . Model-based methods carry out such inference based on prior knowledge of the statistical model relating Y and S, denoted p Y |S . Model-based ML systems reviewed in this chapter combine model-based methods with learning techniques, namely, they tune their mapping of the input Y based on both a labeled training set {s t , y t } nt t=1 as well as some knowledge of the underlying distribution. Such hybrid data-driven model-aware systems can typically learn their mappings from smaller training sets compared to purely model-agnostic DNNs, and commonly operate without full and accurate knowledge of p Y |S , upon which model-based methods are based. We next elaborate on the main strategies of combining ML and model-based techniques, beginning with extreme cases of DNNs that rely solely on data and purely model-based inference algorithms.

Conventional Deep Learning
The conventional application of deep learning is to carry out inference using some standard DNN architecture. This DNN uses the training data to learn how to map a realization of the input Y = y into a predictionŝ. Such highlyparameterized networks can effectively approximate any Borel measurable mapping, as it follows from the universal approximation theorem [13,Ch. 6.4.1]. Therefore, by properly tuning their parameters using a sufficiently large training set, typically using optimization based on some variant of stochastic gradient descent (SGD), one should be able to obtain the desirable inference rule. While standard DNNs structures are highly model-agnostic and are commonly treated as black-boxes, one can still incorporate some level of domain knowledge in the selection of the specific network architecture. For instance, when the input is known to exhibit temporal correlation, architectures based on recurrent neural networks (RNNs) or transformers are known to be preferable. Alternatively, in the presence of spatial patterns, one may prefer to utilize convolutional layers. An additional method to incorporate domain knowledge into a black-box DNN is by pre-processing of the input via, e.g., feature extraction.
Conventional deep learning based on established black-box DNNs is datadriven, i.e, it requires data representing the problem at hand, possibly combined with a very basic level of domain knowledge to select the specific architecture. A major drawback of using such networks, which is particularly relevant in the context of communication systems, is that learning a large number of parameters requires a massive data set to train. In dynamic environments, even when a sufficiently large data set is available, it is difficult to train a model that performs optimally over the whole range of the dynamically changing system. Moreover, online training as the system dynamics change tends to be computationally expensive because of the large number of parameters.

Model-Based Methods
Model-based algorithms carry out inference based on prior knowledge of the underlying statistics relating the input Y and the label S, i.e., p Y |S . A common family of model-based methods is based on iterative algorithms, which allow us to infer with provable performance and controllable complexity in an iterative fashion, as illustrated in Fig. 1.2(a). This generic family of iterative algorithms consists of some input and output processing stages, with an intermediate iterative procedure. The latter can in turn be divided into a model-based computation, namely, a procedure that is determined by p Y |S ; and a set of generic mathematical manipulations. These algorithms vary significantly between different statistical models. For instance, for the symbol detection task, model-based methods such as the Viterbi detector [14] or the BCJR algorithm [15] are valid for finite-memory channels, while for multiple-input multiple-output (MIMO) detection one may utilize the family of interference cancellation methods [16]. Each such algorithm may be specifically tailored to a given scenario, as opposed to black-box DNNs, in which the parameterized inference rule is generic, and the unique characteristics of the scenario at hand are encapsulated in the parameters learned during training.
Model-based techniques do not rely on data to learn their mapping, though data is often used to estimate unknown model parameters. In practice, accurate knowledge of the statistical model relating the observations and the desired information is typically unavailable, and thus applying such techniques commonly requires imposing some assumptions on the underlying statistics, which in some cases reflects the actual behavior, but often do not. In the presence of inaccurate knowledge of p Y |S due to estimation errors or due to enforcing a model that does not fully capture the environment, the performance of model-based methods tends to degrade considerably. This limits the applicability of model-based algorithms in scenarios where p Y |S is unknown, costly to estimate accurately, or too complex to express analytically.

1.2.3
Model-Based Deep Learning by Deep Unfolding Deep unfolding [8,9], also referred to as deep unrolling, is a common strategy to combine deep learning with model-based algorithms. Here, model-based methods are utilized as a form of domain knowledge in designing a DNN architecture, which is trained end-to-end and then used for inference. Unlike the application of conventional black-box DNNs discussed in Section 1.2.1, deep unfolding utilizes a unique DNN structure designed specifically for the task at hand. The main rationale in deep unfolding is to design the network to imitate the operation of a model-based iterative optimization algorithm corresponding to the considered problem. In particular, each iteration of the model-based algorithm is replaced with a dedicated layer with trainable parameters whose structure is based on the operations carried out during that iteration. An illustration of a neural network obtained by unfolding the model-based iterative model of Fig. 1.2(a) is depicted in Fig. 1.2(b), where a network with T layers is designed to imitate T iterations of the optimization method. Once the architecture is fixed, the resulting network is trained in an end-to-end manner as in conventional deep learning.
Deep unfolded networks, in which the iterations consist of trainable parameters, are typically capable of inferring with a smaller number of layers compared to the amount of iterations required by the model-based algorithm. Consequently, even when the model-based algorithm is feasible, processing Y through a trained unfolded DNN is typically faster than applying the iterative algorithm [4]. Furthermore, converting a model-based algorithm into an unfolded deep network can also improve its performance. For example, iterative algorithms based on some relaxed optimization commonly achieve improved accuracy when unfolded into a DNN, due to the ability to learn to overcome the error induced by relaxation in the training stage. The main benefits of deep unfolding over using end-to-end networks stem from its incorporation of domain knowledge in the network architecture. As such, unfolded networks can achieve improved performance at reduced complexity, i.e., when operating with less parameters, compared to conventional end-to-end networks [10]. Nonetheless, deep unfolded networks are highly parameterized DNNs, which often require large data sets for training, though usually not as much as the generic DNNs. Furthermore, deep unfolding typically requires a high level of domain knowledge, such as explicit knowledge of the statistical model p Y |S up to possibly some missing parameters, in order to formulate the optimization algorithm in a manner that can be unfolded.

Model-Based Deep Learning by DNN-Aided Algorithms
The second strategy for combining model-based methods and deep learning, which we refer to as DNN-aided hybrid algorithms, aims at integrating ML into model-based techniques. Such DNN-aided systems mainly utilize conventional model-based methods for inference, while incorporating DNNs to make the re-sultant system more robust and model-agnostic. This approach builds upon the insight that model-based algorithms typically consist of a set of generic manipulations that are determined by the structure of the statistics, e.g., whether it obeys a Markovian structure. Beside these generic manipulations, there are also computations that require actual knowledge of p Y |S , as illustrated in Fig. 1.2(a). Consequently, when one has prior knowledge on the structure of the underlying distribution but not of its actual distribution, ML can be utilized to fill in the missing components required to carry out the algorithm.
In particular, DNN-aided hybrid algorithms start with a model-based algorithm that is suitable for inference when the statistics of p Y |S are available. For instance, symbol detection over finite-memory channels can be carried out accurately and with affordable complexity using either the Viterbi algorithm [14] or the BCJR method [15], assuming p Y |S is known. Then, ML-based techniques, such as dedicated DNNs, are used to estimate only p Y |S from data. These dedicated DNNs can be trained individually, separately from the inference task, or in an end-to-end manner along with the overall algorithm that maps Y into an estimate of S. An illustration of a DNN-aided algorithm obtained by integrating ML into the iterative methods illustrated in Fig. 1.2(a) is depicted in Fig. 1.2(c).
DNN-aided hybrid algorithms have several advantages: First, they use DNNs for specific intermediate tasks, such as computing a conditional probability measure, which are much simpler compared to end-to-end inference. Consequently, relatively simple networks that are trainable using small training sets can be used. Furthermore, once trained the system effectively implements the modelbased algorithm in a data-driven manner without imposing a model on the underlying distribution and estimating its parameters. Concrete examples of DNNaided symbol detection algorithms are detailed in Section 1.3.4.

Deep Symbol Detection
In digital communication systems, the receiver is required to reliably recover the transmitted symbols from the observed channel output. This task is commonly referred to as symbol detection. In this section, we present how the strategies for combining ML and model-based algorithms detailed in the previous section can be applied for data-driven symbol detection. We first formulate the symbol detection in Section 1.3.1, after which we discuss the applications of data-driven receivers based on conventional DNN architectures, deep unfolding, and DNNaided algorithms, in Sections 1.3.2-1.3.4, respectively. For each strategy we begin with the main rationale behind this approach, present at least one concrete example, and discuss its pros and cons. Finally, we numerically compare the data-driven receivers to their model-based counterparts in Section 1.3.5.

The Symbol Detection Problem
To formulate the symbol detection problem, we let S i ∈ S K be the symbol transmitted at time index i ∈ {1, 2, . . . , T } := T . Here, T represents the blocklength and K denotes the number of symbols transmitted at each time instance, e.g., the number of users transmitting simultaneously in the uplink communications channel. Each symbol is uniformly distributed over a set of M constellation points, thus |S| = M . We use Y i ∈ Y N to denote the channel output at time index i, where N represents the number of receive antennas. When both N and K are larger than one, the resulting setup corresponds to MIMO communications. Symbol detection refers to the recovery of S T := {S i } i∈T from the observed Y T := {Y i } i∈T . An illustration of the symbol detection problem using a DNN-aided receiver is depicted in Fig. 1.3. We focus on finite-memory stationary causal channels, where each Y i is given by a stochastic mapping of {S l } i l=i−L+1 , and L is the memory of the channel, assumed to be smaller than the blocklength T . The special case in which L = 1 is referred to as flat or memoryless channel conditions. The conditional probability density function (PDF) of the channel output given its input thus satisfies where the lower-case y i and s i represent the realizations of the random variables Y i and S i , respectively. The fact that the channel is stationary implies that the conditional PDF p Y i|{Sl } i l=i−L+1 does not depend on the index i. The symbol detection mapping that minimizes the error rate is the maximum a-posteriori probability (MAP) rule, given bŷ For the memoryless case L = 1, solving (1.2) reduces to maximizing p S|Y (s|y i ) over s ∈ S K . However, when K is large, as is commonly the case in uplink MIMO systems, solving (1.2) may be computationally infeasible, even when the PDF p S|Y is perfectly known. The application of deep learning for symbol detection thus has two main motivations: the first is to allow symbol detection to operate in a model-agnostic manner, i.e., without requiring knowledge of p S|Y , and the second is to facilitate inference when the computational complexity of (1.2) renders solving it infeasible.

Symbol Detection via Established Deep Networks
The first approach to designing data-driven symbol detectors treats the channel as a black-box and relies on well-known deep learning architectures used in computer vision, speech, and language processing for detection. We now describe how conventional deep neural architectures can be used for symbol detection.

Overview of Design Process
Different DNN architectures have shown promising results for detection and estimation in applications such as image processing [17,18,19], speech recognition [20,21,22], machine translation [23,24,25], and bioinformatics [26,27]. Some of these neural network architectures can be used to design a symbol detector for channels with unknown models using supervised learning. This process typically consists of the following steps: 1. Identify the conventional neural network architectures that are suitable for the channel under consideration, and use these networks as building blocks for designing the detection algorithm. For example, RNNs are more suitable for sequential detection in channels with memory, while convolutional and fully-connected networks are more suitable for memoryless channels. 2. Next, use channel input-output pairs to train the network. Two approaches can be used for training: In the first approach, a model is trained for each channel condition (e.g., each SNR). In the second approach, a large training dataset consisting of various channel conditions is used to train a single neural network detector for detection over a wide-range of channel conditions. The training data can be generated by randomizing the transmitted symbols and generating the corresponding received signal using mathematical models, simulations, experiments, or field measurements. 3. Train the overall resulting network in an end-to-end fashion.
We next demonstrate how this rationale is translated into a concrete datadriven symbol detector architecture for finite-memory channels.

Example: SBRNN for Finite-Memory Channels
The sliding bidirectional RNN (SBRNN) is a sequence detection algorithm for finite-memory channels proposed in [28]. Generally, sequence detection can be performed using RNNs [1], which are well established for sequence estimation in different problems such as neural machine translation [23], speech recognition [2], or bioinformatics [26]. For simplicity, we assume in our description that the input cardinality K is K = 1. The estimated symbol in this case is given bŷ where P RNN is the probability of estimating each symbol based on the DNN model used. One of the main benefits of this detector is that after training, it can perform detection on any data stream as it arrives at the receiver. This is because the observations from previous symbols are summarized as the state of the RNN, which is represented by a vector. Note that the observed signal during the jth transmission, Y j where j > k, may carry information about the kth symbol S k due to the memory of the channel. However, since RNNs are feed-forward only, during the estimation ofŜ k , the observation signal Y j is not considered.
One way to overcome this limitation is by using bidirectional RNNs (BRNNs). In such networks, a sequence of B received signals are fed once in the forward direction into one RNN cell, and fed once in the backward direction into another RNN cell [29], for some fixed B representing the BRNN length. The two outputs are then concatenated and may be passed to more bidirectional layers. A signal whose blocklength T is larger than the BRNN length B is divided into multiple distinct subsequences of length B. Ideally, the B must be at least the same size as the memory length L. However, if this is not known in advance, the BRNN length can be treated as a hyperparameter to be tuned during training. At time instance i belonging to the kth subsequence, the estimated symbol for BRNN is given byŝ To simplify the notation, we usep to denote the M × 1 matrix whose entries are the probability mass function (PMF) P BRNN (s i |y k+B−1 k ) for each s i ∈ S. The BRNN architecture ensures that in the estimation of a symbol, future signal observations are taken into account. During training, blocks of B consecutive transmissions are used for training. Once the network is trained, BRNNs detect the stream of incoming data in fixed blocks of length B, as shown in the top portion of Fig. 1.4. The main drawback here is that the symbols at the end of each block may affect the symbols in the next block, and since each block is treated independently, this relation is not captured in this scheme. Another issue is that the block of B symbols must be received before detection can be performed. The top portion of Figure 1.4 shows this scheme for B = 3.
To overcome these limitations, in the SBRNN proposed in [28], the first B symbols are detected using the BRNN. Then, as each new symbol arrives at the receiver, the subsequence processed by the BRNN slides ahead by one symbol. Let the set K i = {k | k ≤ i ∧ k + B > i} be the set of all valid starting positions for a BRNN detector of length B, such that the detector overlaps with the ith symbol. For example, if B = 3 and i = 4, then k = 1 is not in the set K i since the BRNN detector overlaps with symbol positions 1, 2, and 3, and not the symbol position 4. The estimated PMF corresponding to the ith symbol is given by the weighted sum of the estimated PMFs for each of the relevant windows: The weighted sum coefficients can be set to α k = 1 |Ki| , as we do in the numerical

Summary
Well-known DNN architectures can be trained in an end-to-end manner to perform symbol detection. This approach builds upon the success of existing modelagnostic DNN structures, resulting in symbol detectors operating without any knowledge about the underlying channel models. Furthermore, this strategy allows combining some basic level of domain knowledge in the selection of the architecture as well as the preparation of its input. For example, the SBRNN detector detailed above identifies the BRNN architecture as one that is capable of handling temporal correlation in finite-memory channels, while using a sliding subsequence to overcome some of the limitations of BRNN architectures when applied to different blocks independently. Also, DNNs can well-capture the nonlinearities that may exist in the channel. For example, the SBRNN was used as an autoencoder in [30,31] to achieve state-of-the-art performance over optical channels, outperforming model-based nonlinear equalizers such as Voltera. Finally, these networks can also be computationally more efficient than the optimal maximum-likelihood sequence detector over finite-memory channels, specifically for channels with long memory. The main drawback in using established deep networks for end-to-end symbol detection is that such architectures typically have a very large number of parameters, and thus require massive data sets for training. This renders online training using pilot sequences impractical. Moreover, when they are trained using data from a large set of channel conditions, the resulting network will not perform optimally for each of those channel conditions individually. Furthermore, even when the data set is extremely large and diverse, it is not likely to capture all expected channel conditions. Finally, conventional DNN architectures are treated as black-boxes, and are in general not interpretable, making it difficult to come up with performance guarantees.

Symbol Detection via Deep Unfolding
Unlike conventional DNNs, which utilize established architectures, in deep unfolding the network structure is designed following a model-based algorithm. We next describe how this model-based ML technique can be applied for symbol detection, and detail a concrete example for flat Gaussian MIMO channels.

Overview of Design Process
Deep unfolding is a method for converting an iterative algorithm into a DNN by designing each layer of the network to resemble a single iteration. As such, the rationale in applying deep unfolding consists of the following steps: 1. Identify an iterative optimization algorithm that is useful for the problem at hand. For instance, recovering the MAP symbol detector for flat MIMO channels can be tackled using various iterative optimization algorithms, such as projected gradient descent, unfolded into DetNet [32], as described in the sequel. 2. Fix a number of iterations in the optimization algorithm. 3. Design the layers to initiate the operation of each iteration in a trainable fashion, as illustrated in Fig. 1.2. 4. Train the overall resulting network in an end-to-end fashion.
We next demonstrate how this rationale is translated into a concrete data-driven symbol detector architecture for flat MIMO channels, i.e., (1.1) with L = 1.

Example: DetNet for Flat MIMO Channels
DetNet is a deep learning based symbol detector proposed in [32] for flat Gaussian MIMO channels. To formulate DetNet, we first detail the specific channel model for which it is designed, and then show how it is obtained by unfolding the projected gradient descent method for recovering the MAP estimate.

Flat Gaussian MIMO Channel
As we focus on stationary memoryless channels, we drop the subscript i representing the time instance, and write the input-output relationship of a flat Gaussian MIMO channel as

Project Gradient Descent Optimization
While directly solving (1.7) involves an exhaustive search over the 2 K possible symbol combinations, it can be tackled with affordable computational complexity using the iterative projected gradient descent algorithm. Let P S (·) denote the projection into the S operator, which for BPSK constellations is the sign function. The projected gradient descent iteratively refines its estimate, which at iteration index q + 1 is obtained recursively aŝ where η q denotes the step size at iteration q, andŝ 0 is set to some initial guess.

Unfolded DetNet
DetNet unfolds the projected gradient descent iterations in (1.8) into a DNN, which learns to carry out this optimization procedure from data. To formulate DetNet, we first fix a number of iterations Q. Next, we design a DNN with Q layers, where each layer imitates a single iteration of (1.8) in a trainable manner. In particular, DetNet builds upon the observation that each projected gradient descent iteration consists of two stages: gradient descent computation, i.e., s q − η q H T y + η q H T Hŝ q , and projection, namely, applying P S (·). Therefore, each unfolded iteration is represented as two sub-layers: The first sub-layer learns to compute the gradient descent stage by treating the step-size as a learned parameter and applying a conventional fully-connected layer with ReLU activation to the obtained value. For iteration index q, this results in in which {W 1,q , b 1,q , δ 1,q , δ 2,q } are learnable parameters. The second sub-layer learns the projection operator by approximating the sign operation with a soft sign activation proceeded by a fully-connected layer, leading tô Here, the learnable parameters are {W 2,q , b 2,q }. The resulting deep network is depicted in Fig. 1.5, in whichŝ 0 is set to some initial guess, and the output after Q iterations, denotedŝ Q , is used as the estimated symbol vector by taking the sign of each element.
be the trainable parameters of DetNet a . To tune θ, the overall network is trained in an end-to-end manner to minimize the empirical weighted 2 norm loss over its intermediate layers. In particular, by letting {s t , y t } nt t=1 denote the training set consisting of channel outputs and their corresponding transmitted symbols, the loss function used for training DetNet is given by whereŝ q (y t ; θ) is the output of the qth layer of DetNet with parameters θ and input y t . This loss measure accounts for the interpretable nature of the unfolded network, in which the output of each layer is a further refined estimate of S.

Summary
Deep unfolding incorporates model-based domain knowledge to obtain a dedicated DNN design, which follows an iterative optimization algorithm. Compared to the conventional DNNs discussed in the previous section, unfolded networks are typically interpretable, and tend to have a smaller number of parameters, and can thus be trained quicker [10]. Nonetheless, these deep networks are still highly parameterized, and require a large volume of training data. For instance, DetNet is trained in [32] using approximately 250 million labeled samples. One of the key properties of unfolded networks is their reliance on model knowledge. For example, the unfolded receiver must know that the channel input-output relationship takes the form (1.6) in order to formulate the projected gradient iterations (1.8), which in turn are unfolded into DetNet. The model-awareness of deep unfolding has its advantages and drawbacks. When the model is accurately known, deep unfolding essentially incorporates it into the DNN architecture, as opposed to conventional DNNs that must learn this from a The formulation of DetNet in [32] includes an additional sub-layer in each iteration intended to further lift its input into higher dimensions and introduce additional trainable parameters, as well as reweighing of the outputs of subsequent layers. As these operations do not follow directly from unfolding the projected gradient descent method, they are not included in the description here.
data. However, this approach does not exploit the model-agnostic nature of deep learning, and thus may achieve degraded performance when the true channel conditions deviate from the model assumed in design, e.g., (1.6). In particular, a key advantage of deep unfolding over the model-based optimization algorithm is in inference speed. For instance, DetNet requires fewer layers to reliably detect compared to the number of iterations required for projected gradient descent to converge.
Another important advantage of unfolded networks is their ability to improve the accuracy compared to the iterative optimization algorithm from which they originate. In particular, the set {±1} K over which the optimization problem (1.7) is formulated is not convex, and thus projected gradient descent is not guaranteed to recover its solution, regardless of the number of iterations. By unfolding the application of projected gradient descent for solving (1.7) into a DNN with trainable parameters, the resulting network is often able to overcome this difficulty and converge to the true solution of the optimization problem when properly trained, despite the non-convexity. Finally, in the context of receiver design, most unfolded networks to date, including DetNet as well as OAMP-net [33] which unfolds the orthogonal approximate message passing optimization algorithm, require channel state information (CSI). For example, knowledge of the matrix H is utilized in the architecture depicted in Fig. 1.5. This implies that additional mechanisms for estimating the channel must be incorporated into the receiver architecture [34]. Nonetheless, while the aforementioned deep unfolding based receivers require CSI, unfolded networks can be designed without such knowledge [9]. For example, one can unfold the optimization algorithm assuming CSI is available, and then treat H which appears in the unfolded network as part of its trainable parameters.

Symbol Detection via DNN-Aided Algorithms
DNN-aided hybrid algorithms combine domain knowledge in the form of a modelbased inference algorithm for the problem at hand. This strategy allows the design of model-based ML systems with varying levels of domain knowledge, in which deep learning is used to robustify and remove model-dependence of specific components of the algorithm. In the following we first review the rationale when designing DNN-aided symbol detectors, after which we detail three concrete examples arising from different symbol detection algorithms.

Overview of Design Process
DNN-aided algorithms aim to carry out model-based inference methods in a data-driven fashion. These hybrid systems thus utilize deep learning not for the overall inference task, but for robustifying and relaxing the model-dependence of established model-based inference algorithms. Consequently, the design of DNNaided hybrid systems consists of the following steps: 1. First, a proper inference algorithm is chosen. In particular, the domain knowledge is encapsulated in the selection of the algorithm that is learned from data. For example, the Viterbi algorithm is a natural candidate for symbol detection over finite-memory channels when seeking a symbol detector capable of operating in real-time, or alternatively, the BCJR scheme is the suitable choice for carrying out MAP inference over such channels. When designing receivers for flat MIMO channels, interference cancellation methods may be the preferable algorithmic approach for symbol detection. We show how these methods are converted into DNN-aided algorithms in the sequel. 2. Once a model-based algorithm is selected, we identify its model-specific computations, and replace them with dedicated compact DNNs. 3. The resulting DNNs can be either trained individually, or the overall system can be trained in an end-to-end manner.
Since the implementation of DNN-aided algorithms highly varies with the selection of the learned model-based method, we next present three concrete examples in the context of symbol detection: ViterbiNet, which learns to carry out Viterbi detection [14]; BCJRNet, which implements the BCJR algorithm of [15] in a data-driven fashion; and DeepSIC, which is based on the soft iterative interference cancellation methods for MIMO symbol detection [35].

Example: ViterbiNet for Finite-Memory Channels
ViterbiNet proposed in [11] is a data-driven implementation of the Viterbi detection algorithm [14], which is one of the most common workhorses in digital communications. This DNN-aided symbol detection algorithm is suitable for finite-memory channels of the form (1.1), without requiring prior knowledge of the channel conditional distributions p Y i|{Sl} i l=i−L+1 . For simplicity, we assume in our description that the input cardinality K is K = 1. As a preliminary step to presenting ViterbiNet, we now briefly review conventional model-based Viterbi detection.

The Viterbi Algorithm
The Viterbi algorithm recovers the maximum likelihood sequence detector, i.e., s T y T := arg max (1.12) Using (1.1) the optimization problem (1.12) becomeŝ (1.14) The optimization problem (1.14) can be solved recursively using dynamic programming, by iteratively updating a path cost c i (s) for each states ∈ S L . The resulting scheme, known as the Viterbi algorithm, is given below as Algorithm 1, and illustrated in Fig. 1.6(a).
Update cost and path via The Viterbi algorithm has two major advantages: 1) It solves (1.12) at a computational complexity that is linear in the blocklength T . For comparison, the computational complexity of solving (1.12) directly grows exponentially with T ; 2) The algorithm produces estimates sequentially during run-time. In particular, while in (1.12) the estimated outputŝ T is computed using the entire received block y T , Algorithm 1 computesŝ i once y i+L−1 is received.

ViterbiNet
ViterbiNet proposed in [11] learns to implement the Viterbi algorithm from data in a model-agnostic manner. Following the rationale of DNN-aided algorithms, this is achieved by identifying the model-based components of the algorithm, which for Algorithm 1 boils down to the computation of the log-likelihood function log p Y i|Si (y i |s). Once this quantity is computed for eachs ∈ S L , the Viterbi algorithm only requires knowledge of the memory length L. This requirement is much easier to satisfy compared to full CSI.
Since the channel is stationary, it holds that the log-likelihood function depends only on the realizations of y i and ofs, and not on the time index i.  Therefore, to implement Algorithm 1 in a data-driven fashion, ViterbiNet replaces the explicit computation of the log-likelihoods with an ML-based system that learns to evaluate this function from training data. In this case, the input of the system is the channel output realization y i and the output is an estimate of log p Y i|Si (y i |s) for eachs ∈ S L . The rest of the Viterbi algorithm remains intact, and the detector implements Algorithm 1 using the learned log-likelihoods. The proposed architecture is illustrated in Fig. 1.6(b).
Two candidate architectures are considered for learning to compute the log likelihood, one based on classification networks and one using density estimation networks. Learned likelihood using classification networks: Since y i is given and may take continuous values while the desired variables take discrete values, a natural approach to evaluate p Y i|Si (y i |s) for eachs ∈ S L is to estimate pS i|Y i (s|y i ) and then use Bayes rule to obtain (1.17) A parametric estimate of pS i|Y i (s|y i ), denotedP θ (s|y i ), is obtained for each s ∈ S L by training classification networks with softmax output layers to minimize the cross entropy loss. Here, for a labeled set {s t , y t } nt t=1 , the loss function is In general, the marginal PDF of Y i can be estimated from the training data using mixture density estimation via, e.g., expectation minimization (EM) [36,Ch. 2], or any other finite mixture model fitting method. However, obtaining an accurate density estimation becomes challenging when Y i is high-dimensional. Since p Y i (y i ) does not depend on the variables, setting p Y i (y i ) ≡ 1 does not affect the decisions in Algorithm 1 due to the arg min arguments, which are invariant to scaling the conditional distribution by a term that does not depend ons. The resulting structure is illustrated in Fig. 1.7(a). Learned likelihood using conditional density networks: An additional strategy is to directly estimate the conditional p Y i|Si (y i |s) from data. This can be achieved using conditional density estimation networks [37,38] that are specifically designed to learn such PDFs, or alternatively, using normalizing flow networks to learn complex densities [39]. For example, mixture density networks [37] model the conditional PDF p Y i|Si (y i |s) as a Gaussian mixture, and train a DNN to estimate its mixing parameters, mean values, and covariances, denoted w k,θ (s), µ k,θ (s) and Σ k,θ (s), respectively, by maximizing the likelihood P θ (y i |s) = k w k,θ (s)N y i |µ k,θ (s), Σ k,θ (s) , as illustrated in Fig. 1.7(b).
Both approaches can be utilized for learning to compute the likelihood in ViterbiNet. When the channel outputs are high-dimensional, i.e., N is large, directly learning the conditional density is difficult and likely to be inaccurate. In such cases, the classification-based architecture, which avoids the need to explicitly learn the density, may be preferable. When the state cardinality M L is large, conditional density networks are expected to be more reliable. However, the Viterbi algorithm becomes computationally infeasible when M L grows, regardless of whether it is implemented in a model-based or data-driven fashion, making ViterbiNet non-suitable for such setups.

Example: BCJRNet for Finite-Memory Channels
Factor graph methods, such as the sum-product algorithm, exploit the factorization of a joint distribution to efficiently compute a desired quantity [40]. In particular, the application of the sum-product algorithm for the joint input-output distribution of finite-memory channels allows for computing the MAP rule, an operation whose burden typically grows exponentially with the block size, with complexity that only grows linearly with T . This instance of the sum-product algorithm is exactly the BCJR detector proposed [15]. In the following we show how the BCJR method can be extended into the DNN-aided BCJRNet. As in our description of ViterbiNet, we again focus on the case of K = 1, and begin by presenting the model-based BCJR algorithm.

The BCJR Algorithm
The BCJR algorithm computes the MAP rule in (1.2) for finite-memory channels with complexity that grows linearly with the block size. To formulate this method, we recall the definition ofS i , and define the function Combining (1.19) and (1.1), we obtain a factorizable expression of the joint distribution p Y T ,S T (·), given by The factorizable expression of the joint distribution (1.20) implies that it can be represented as a factor graph with T function nodes {f (y i ,s i ,s i−1 )}, in which {s i } T −1 i=2 are edges while the remaining variables are half-edges b . Using its factor graph representation, one can compute the joint distribution of S T and Y T by recursive message passing along this factor graph. In particular, for i = 1, 2, . . . , k. Similarly, the backward messages are for i = T − 1, T − 2, . . . , k + 1. This message passing is illustrated in Fig. 1.8. The ability to compute the joint distribution in (1.21) via message passing results in the MAP detector in (1.2), an operation whose burden typically grows exponentially with the block size, with complexity that only grows linearly with T . This is achieved by noting that the MAP estimate satisfieŝ Here we use Forney style factor graphs [41], where variables are represented as edges or half-edges. However, it is also possible to represent variables as variables notes. for each i ∈ T , where the summands can be computed recursively. When the block size T is large, the messages may tend to zero, and are thus commonly scaled [42], e.g., ← − µS i (s) is replaced with γ i ← − µS i (s) for some scale factor that does not depend ons, and thus does not affect the MAP rule. The BCJR algorithm is summarized as Algorithm 2.

BCJRNet
BCJRNet is a receiver method that learns to implement MAP detection from labeled data. BCJRNet exploits the fact that in order to implement Algorithm 2, one must be able to specify the factor graph representing the underlying distribution. In particular, the stationarity assumption implies that the complete factor graph is encapsulated in the single function f (·) (1.19) regardless of the block size T . Building upon this insight, BCJRNet utilizes DNNs to learn the mapping carried out at the function node separately from the inference task. The resulting learned stationary factor graph is then used to recover {S i } by message passing, as illustrated in Fig. 1.9. As learning a single function node is expected to be a simpler task compared to learning the overall inference method for recovering S T from Y T , this approach allows using relatively compact DNNs, which in turn can be learned from a relatively small set of labeled data. Furthermore, the learned function node describes the factor graph for different values of T . When the learned function node is an accurate estimate of the true one, BCJR-Net effectively implements the MAP detection rule (1.2), and thus approaches the minimal probability of error. The function node that encapsulates the factor graph of stationary finitememory channels is given in (1.19). The formulation in (1.19) implies that it can be estimated by training an ML-based system to evaluate p Y i|Si (·) from which the corresponding function node value is obtained via (1.19). Once the factor graph representing the channel is learned, symbol recovery is carried out using Algorithm 2. As the mapping of Algorithm 2 is invariant to scaling f (y i ,s i ,s i−1 ) with some factor that does not depend on the states, it follows that a parametric estimate of the function f (·), denotedf θ (·), can be obtained using the same networks utilized for learning the log-likelihood in ViterbiNet. Specifically, the learned log-likelihood is used in (1.19) to obtainf θ (·). The resulting receiver, referred to as BCJRNet, thus implements BCJR detection in a data-driven manner.

BCJRNet vs. ViterbiNet
The same DNN architecture can be applied, once trained, to carry out multiple inference algorithms in a hybrid model-based/data-driven manner, including the BCJR scheme (as BCJRNet) as well as the Viterbi algorithm (via ViterbiNet). Since both BCJRNet and ViterbiNet utilize the same learned models, one can decide which inference system to apply, i.e., BCJRNet or ViterbiNet, by considering the differences in the algorithms from which they are derived, i.e., the BCJR algorithm and the Viterbi algorithm, respectively. The main advantages of Algorithm 1 over Algorithm 2, and thus of ViterbiNet over BCJRNet, are its reduced complexity and real-time operation. In particular, both algorithms implement recursive computations, involving |S| L evaluations for each sample, and thus the complexity of both algorithms grows linearly with the block size T . Nonetheless, the Viterbi scheme computes only a forward recursion and can thus provide its estimations in real time within a given delay from each incoming observation, while the BCJR scheme implements both forward and backward recursions, and can thus infer only once the complete block is observed, while involving twice the computations carried out by the Viterbi detector.
The main advantage of Algorithm 2 over Algorithm 1, i.e., of BCJRNet over ViterbiNet, stems from the fact that it implements the MAP rule (1.2), which minimizes the error probability. The Viterbi algorithm is designed to compute the maximum likelihood sequence detector, i.e., For a given realization Y T = y T , the function p S T |Y T s T |y T maximized by the sequence-wise detector in (1.26) is a joint distribution measure. The functions {p Si|Y T s i |y T } T i=1 , which are the individually maximized by the symbol-wise MAP rule computed by the BCJR scheme (1.2), are the marginals of the aforementioned joint distribution. Furthermore, given Y T = y T the elements of S T are statistically dependent in finite-memory channels. As a a result, the maxima of the joint distribution p S T |Y T is not necessarily the individual maximas of each of its marginals, i.e., the elements of the vectorŝ T y T in (1.25) are not necessarily the symbol-wise MAP estimates {ŝ i y T } T i=1 in (1.2). To conclude, the Viterbi algorithm does not implement the symbol-wise MAP even in the presence of equal priors. This explains the difference in their performance, since, unlike the BCJR scheme, the Viterbi algorithm does not minimize the error probability.

Example: DeepSIC for Flat MIMO Channels
DeepSIC proposed in [43] is a DNN-aided hybrid algorithm that is based on the iterative soft interference cancellation (SIC) method [35] for symbol detection in flat MIMO channels. However, unlike its model-based counterpart, and alternative deep MIMO receivers such as DetNet, it is not tailored for linear Gaussian channels of the form (1.6). The only assumption required is that the channel is memoryless, i.e., L = 1, and thus we drop the time index subscript in this example. As in our previous DNN-aided examples, we first review iterative SIC, after which we present its DNN-aided implementation. Substituting the channel output y into (1.27), the realization of the interference canceled Z k , is obtained. To implement soft decoding, it is assumed thatW )+W obeys a zero-mean Gaussian distribution, independent of S k , and that its covariance is given by ΣW (q) where σ 2 w is the noise variance. Combining this assumption with (1.27), the conditional distribution of Z (q) k given S k = α j is multivariate Gaussian with mean value h k α j and covariance k ) for each α j ∈ S under the above assumptions. Consequently, the conditional distribution of S k given Y is approximated from the conditional distribution of Z (q) k given S k via Bayes theorem. Since the symbols are equiprobable, this estimated conditional distribution is computed as (1.28) After the final iteration, the symbols are decoded by taking the symbol that maximizes the estimated conditional distribution for each user, i.e., The overall joint detection scheme is summarized as Algorithm 3. The initial estimates {p (0) k } K k=1 can be arbitrarily set. For example, these may be chosen based on a linear separate estimation of each symbol for y, as proposed in [35].

Algorithm 3: Iterative Soft Interference Cancellation Algorithm [35]
Init: Set q = 1, and generate an initial guess of {p Iterative SIC has several advantages compared to both joint decoding as well as separate decoding: In terms of computational complexity, it replaces the joint exhaustive search over S K , required by the MAP decoder, with a set of computations carried out separately for each user. Hence, its computational complexity only grows linearly with the number of users [16], making it feasible with large values of K. Unlike conventional separate decoding, in which the symbol of each user is recovered individually while treating the interference as noise, the iterative procedure refines the separate estimates sequentially, and the usage of soft values mitigates the effect of error propagation. Algorithm 3 is thus capable of approaching the performance of the MAP detector, which is only feasible for small values of K.

DeepSIC
Iterative SIC is specifically designed for linear channels of the form (1.6). In particular, the interference cancellation in Step 2 of Algorithm 3 requires the contribution of the interfering symbols to be additive. Furthermore, it requires accurate CSI. To circumvent these limitations in the model-based approach, the DNN-aided DeepSIC learns to implement the iterative SIC from data.
Architecture: DeepSIC builds upon the observation that iterative SIC can be viewed as a set of interconnected basic building blocks, each implementing the two stages of interference cancellation and soft decoding, i.e., Steps 2-3 of Algorithm 3. While the high level architecture in Fig. 1.10 is ignorant of the underlying channel model, the basic building blocks are channel-model-dependent. In particular, interference cancellation requires the contribution of the interference to be additive, i.e., a linear model channel as in (1.6), as well as full CSI, in order to cancel the contribution of the interference. Soft decoding requires complete knowledge of the channel input-output relationship in order to estimate the conditional probabilities via (1.28).
Although each of these basic building blocks consists of two sequential procedures that are completely channel-model-based, we note that the purpose of these computations is to carry out a classification task. In particular, the kth building block of the qth iteration, k ∈ K, q ∈ Q, producesp (q) k , which is an estimate of the conditional PMF of S k given Y = y based on {p (q−1) l } l =k . Such computations are naturally implemented by classification DNNs, e.g., fully-connected networks with a softmax output layer. Embedding these ML-based conditional PMF computations into the iterative SIC block diagram in Fig. 1.10 yields the overall receiver architecture depicted in Fig. 1.11. The initial estimates {p can be set to represent a uniform distribution, i.e., p (0) k j ≡ 1 M . A major advantage of using classification DNNs as the basic building blocks in Fig. 1.11 stems from the fact that such ML-based methods are capable of accurately computing conditional distributions in complex non-linear setups without requiring a-priori knowledge of the channel model and its parameters. Consequently, when these building blocks are trained to properly implement their classification task, the receiver essentially realizes iterative SIC for arbitrary channel models in a data-driven fashion.
Training Methods: In order for the DNN-aided receiver structure of Fig. 1.11 to reliably implement joint decoding, its building block classification DNNs must be properly trained. Here, we consider two possible approaches to train the receiver based on a set of n t pairs of channel inputs and their corresponding outputs, denoted {s t , y t } nt t=1 : End-to-end training, and sequential training. End-to-end training: The first approach jointly trains the entire network, i.e., all the building block DNNs. Since the output of the deep network is the set of conditional distributions {p is used to estimate S k , we use the sum cross entropy as the training objective. Let θ be the network parameters, andp (Q) k (y, α; θ) be the entry ofp (Q) k corresponding to S k = α when the input to the network parameterizd by θ is y. The sum cross entropy loss is Training the receiver in Fig. 1.11 in an end-to-end manner based on the loss (1.30) jointly updates the coefficients of all the K · Q building block DNNs. For a large number of users, training so many parameters simultaneously is expected to require a large labeled set.
Sequential training: To allow the network to be trained with a reduced number of training samples, we note that the goal of each building block DNN does not depend on the iteration index: The kth building block of the qth iteration outputs a soft estimate of S k for each q ∈ Q. Therefore, each building block DNN can be trained individually, by minimizing the conventional cross entropy loss. To formulate this objective, let θ } represent the estimated probabilities associated with y i computed at the previous iteration. The problem with training each DNN individually is that the soft estimates {p Sequential training uses the n t input-output pairs to train each DNN individually. Compared to the end-to-end training that utilizes the training samples to learn the complete set of parameters, which can be quite large, sequential training uses the same data set to learn a significantly smaller number of parameters, reduced by a factor of K · Q, multiple times. Consequently, this approach is expected to require much fewer training samples, at the cost of a longer learning procedure for a given training set, due to its sequential operation, and possible performance degradation as the building blocks are not jointly trained.
Summary DNN-aided algorithms implement hybrid model-based/data-driven inference by integrating ML into established model-based methods. As such, it is particularly suitable for digital communications setups, in which a multitude of reliable model-based algorithms exist, each tailored to a specific structure. The implementation of these techniques in a data-driven fashion thus has three main advantages as a model-based ML strategy: First, when properly trained, the resulting system effectively implements the model-based algorithm from which it originated, thus benefiting from its proven performance and controllable complexity, while being robust to CSI uncertainty and operable in complex environments, due to the usage of DNNs. This behavior is numerically illustrated in the simulation study detailed in Section 1.3.5.
Second, the fact that DNN-aided algorithms use ML tools as intermediate components in the overall end-to-end inference tasks allows the use of relatively compact networks which can be trained with small training sets. Even when the overall system consists of a large set of DNNs, as is the case in DeepSIC, their interpretable operation which follows from the model-based method facilitates their training with small data sets, e.g., via sequential training techniques.
Finally, DNN-aided algorithms can utilize different levels of domain knowledge, depending on what prior information one has on the problem at hand. For example, BCJRNet requires only prior knowledge that the channel has finite memory to learn to carry out MAP detection from data. When additional domain knowledge is available, such as an underlying stationarity or some partial CSI, it can be incorporated into the number and structure of the learned function nodes, further reducing the number of training data required to tune the receiver. The resulting ability of DNN-aided symbol detectors to adapt with small training sets can be exploited to facilitate channel tracking via periodic re-training using existing pilots and other forms of structures present in digital communications protocols, as demonstrated in [11,43].

Numerical Study
In this section, we present a numerical study of the aforementioned symbol detection mechanisms. We begin with considering finite-memory channels, for which we evaluate the data-driven SBRNN receiver, ViterbiNet, and BCJRNet, comparing them to the model-based detection methods for such channels. Then we consider memoryless MIMO channels, where we compare the data-driven DetNet and DeepSIC to model-based detection.

Finite-Memory Channel
We first numerically evaluate the performance of the DNN-aided ViterbiNet and BCJRNet, and compare this performance to that of the conventional modelbased Viterbi algorithm and BCJR detector, as well as to that of the SBRNN receiver detailed in Section 1.3.2. Both ViterbiNet and BCJRNet are implemented using the classification architecture in Fig. 1.7(a) with three fully-connected layers: a 1 × 100 layer followed by a 100 × 50 layer and a 50 × 16(= |M | L ) layer, using intermediate sigmoid and ReLU activation functions, respectively. For the SBRNN receiver, we use BRNN length of B = 10 with 3 layers of LSTM cell blocks of size 100, and a dropout rate of 0.1. The networks are trained using 5000 training samples, which is of the same order and even smaller compared to typical preamble sequences in wireless communication systems. We consider two finite-memory channels: An additive white Gaussian noise (AWGN) channel and a Poisson channel, both with memory length of L = 4. For the AWGN channel, we let W [i] be a zero-mean unit variance AWGN independent of S[i], and let h(γ) ∈ R L be the channel vector obeying an exponentially decaying profile (h) τ e −γ(τ −1) for γ > 0. The input-output relationship is given by where ρ > 0 represents the SNR. The channel input is randomized from a BPSK constellation, i.e., S = {−1, 1}. For the Poisson channel, the symbols represent on-off keying, namely, S = {0, 1}, and the channel output Y [i] is generated via where P(λ) is the Poisson distribution with parameter λ > 0, and X ∼ f (X) indicates that the random variable X is distributed according to f (X). For each channel, we numerically compute the symbol error rate (SER) for different values of the SNR parameter ρ. For each SNR ρ, the SER values are averaged over 20 different channel vectors h(γ), obtained by letting γ vary in the range [0. 1,2]. For comparison, we numerically compute the SER of the Viterbi and BCJR algorithms. In order to study the resiliency of the data-driven detectors to inaccurate training, we also compute the performance when the receiver only has access to a noisy estimate of h(γ), and specifically, to a copy of h(γ) whose entries are corrupted by i.i.d. zero-mean Gaussian noise with variance σ 2 e . In particular, we use σ 2 e = 0.  13, we note that the performance of the data-driven receivers approaches that of their corresponding CSI-based counterparts. We also observe that the SBRNN receiver, which was shown in [28] to approach the performance of the CSI-based Viterbi algorithm when sufficient training is provided, is outperformed by ViterbiNet and BCJRNet here due to the small training set size. These results demonstrate that our DNN-aided detectors, which use compact DNN structures embedded into model-based algorithms, require significantly less training compared to symbol detectors based on using established DNNs for end-to-end inference.
In the presence of CSI uncertainty, it is observed in Figs. 1.12-1.13 that both ViterbiNet and BCJRNet significantly outperform the model-based algorithms from which they originate. In particular, when ViterbiNet and BCJRNet are trained with a variety of different channel conditions, they are still capable of achieving relatively good SER performance under each of the channel conditions for which they are trained, while the performance of the conventional Viterbi and BCJR algorithms is significantly degraded in the presence of imperfect CSI. While the SBRNN receiver is shown to be more resilient to inaccurate CSI compared to the Viterbi and BCJR algorithms, it is outperformed by ViterbiNet and BCJRNet with the same level of uncertainty, and the performance gap is more notable in the AWGN channel.
Finally, we evaluate the application of ViterbiNet and BCJRNet for practical channel models. To that aim, we generate 10 realizations from the established COST2100 model [44], which is a widely used model for current cellular communication channels. In particular, we use the semi-urban 300MHz line-of-sight configuration evaluated in [45] with a single antenna element. The channel output is corrupted by AWGN, and the symbol detectors operate assuming that the channel has L = 4 taps. The remaining simulation parameters are the same as those used in Fig. 1.12. The results, depicted in Fig. 1.14, demonstrate that the ability of ViterbiNet and BCJRNet to approach their model-based counterparts with perfect CSI, as well as achieve improved performance in the presence of CSI uncertainty, holds for practical channel models.

Memoryless MIMO Channel
Next, we numerically compare DeepSIC and DetNet for symbol detection in memoryless MIMO channels. In the implementation of the DNN-based building blocks of DeepSIC, we used a different fully-connected network for each training method: For end-to-end training, where all the building blocks are jointly trained, we used a compact network consisting of a (N + K − 1) × 60 layer followed by ReLU activation and a 60 × M layer. For sequential training, which sequentially adapts subsets of the building blocks and can thus tune more parameters using  Figure 1.14 Symbol error rate of different receiver structures for the AWGN channel with taps generated from the COST2100 model.
the same training set (or, alternatively, requires a smaller training set) compared to end-to-end training, we used three fully-connected layers: An (N +K −1)×100 first layer, a 100 × 50 second layer, and a 50 × M third layer, with a sigmoid and a ReLU intermediate activation functions, respectively. In both iterative SIC as well as DeepSIC, we set the number of iterations to Q = 5. Following [32], DetNet is implemented with Q = 90 layers with a hidden sub-layer size of 8K. The data-driven receivers are trained with a relatively small data set of 5000 training samples, and tested over 20000 symbols. We first consider a linear AWGN channel as in (1.6) with a relatively small K. Recall that iterative SIC as well as DetNet are all designed for such channels. Consequently, the following study compares the performance of DeepSIC and DetNet to that of the model-based iterative SIC as well as the MAP rule (1.2) in a scenario for which all schemes are applicable. The model-based MAP and iterative SIC detectors, as well as DetNet, all require CSI, and specifically, accurate knowledge of the channel matrix H. DeepSIC operates without a-priori knowledge of the channel model and its parameters, learning the decoding mapping from a training set sampled from the considered input-output relationship. In order to compare the robustness of the detectors to CSI uncertainty, we also evaluate them when the receiver has access to an estimate of H with entries corrupted by i.i.d. additive Gaussian noise whose variance is given by σ 2 e times the magnitude of the corresponding entry, where σ 2 e > 0 is referred to as the error variance. For DeepSIC, which is model-invariant, we compute the SER under CSI uncertainty by using a training set whose samples are randomized from a channel in which the true H is replaced with its noisy version.
We simulate the 6 × 6 linear Gaussian channel, i.e., K = 6 users and N =  6 receive antennas. The symbols are randomized from a BPSK constellation, namely, S = {−1, 1} and M = |S| = 2. The channel matrix H models spatial exponential decay, and its entries are given by (H) i,k = e −|i−j| , for each i ∈ {1, . . . , N }, k ∈ K. For each channel, the SER of the receivers is evaluated for both perfect CSI, i.e., σ 2 e = 0, as well as CSI uncertainty, for which we use σ 2 e = 0.1. The evaluated SER versus the SNR, defined as 1/σ 2 w , is depicted in Fig. 1.15.
Observing Fig. 1.15, we note that the performance of DeepSIC with end-toend training approaches that of the model-based iterative SIC algorithm, which is within a small gap of the MAP performance. This demonstrates the ability of DeepSIC to implement iterative SIC in a data-driven fashion. The sequential training method, whose purpose is to allow DeepSIC to train with smaller data sets compared to end-to-end training, also achieves SER values comparable to iterative SIC. DetNet, which trains a large number of parameters in an end-toend fashion, requires 100 times more training to approach such performance. In the presence of CSI uncertainty, DeepSIC is observed to substantially outperform the model-based iterative SIC and MAP receivers, as well as DetNet operating with a noisy version of H and trained with a hundred times more samples. In particular, it follows from Fig. 1.15 that a relatively minor error of variance σ 2 e = 0.1 severely deteriorates the performance of the model-based methods, while DeepSIC is hardly affected by the same level of CSI uncertainty.
Next, we consider a Poisson channel. We use K = 4 and N = 4. Here, the symbols are randomized from an on-off keying for which S = {0, 1}. The entries 10   As DetNet is designed for linear Gaussian channels, DeepSIC is the only datadriven receiver evaluated for this channel. The achievable SER of DeepSIC versus SNR under both perfect CSI as well as CSI uncertainty with error variance σ 2 e = 0.1 is compared to the MAP and iterative SIC detectors in Fig. 1.16. Observing Fig. 1.16, we again note that the performance of DeepSIC is only within a small gap of the MAP performance with perfect CSI, and that the data-driven receiver is more robust to CSI uncertainty compared to the model-based MAP. In particular, DeepSIC with sequential training, which utilizes a deeper network architecture for each building block, outperforms here end-to-end training with basic two-layer structures for the conditional distribution estimation components. We conclude that under such non-Gaussian channels, more complex DNN models are required to learn to cancel interference and carry out soft detection accurately. Furthermore, iterative SIC, which is designed for linear Gaussian channels (1.6) where interference is additive, achieves very poor performance when the channel model is substantially different from (1.6). These results demonstrate the ability of DeepSIC to achieve excellent performance through learning from data for statistical models where model-based interference cancellation is effectively inapplicable.

Summary
Deep learning brings forth capabilities that can substantially contribute to future communications systems in tackling some of their expected challenges. In particular digital communications systems can significantly benefit from properly harnessing the power of deep learning and its model-agnostic nature. A successful integration of deep learning into communication devices can thus pave the way to reliable and robust communications in various setups, including environments where accurate statistical channel models are scarce or costly to obtain. However, digital communications setups are fundamentally different from applications in which deep learning has been extremely successful to date, such as computer vision. In particular, digital communication exhibit an extremely large number of possible outputs, as these outputs grow exponentially with the modulation order and the block length. They also have channel conditions that vary dynamically, and require low computation complexity when used on small battery-powered devices. Consequently, in order to achieve the potential benefits of DNN-aided communications, researchers and system designers must go beyond the straight-forward application of DNNs designed for computer vision and natural language processing. A candidate strategy to utilize DNNs while accounting for the unique characteristics of digital communications setups, as well as the established knowledge of model-based communication methods accumulated over the last decades, is based on model-based ML, as detailed in this chapter. We reviewed two main strategies for combining data-driven deep learning with model-based methods for digital communications, as well as discussing the need for such hybrid schemes due to the shortcomings of the extreme cases of purely data-driven and solely model-based methods. For each strategy, we presented the main steps in the design of the data-driven systems, and provided concrete examples, all in the context of the basic communication task of symbol detection. We first discussed how established DNN architectures can be utilized as symbol detectors, presenting the SBRNN receiver of [28] as an example. Then we detailed how the framework of deep unfolding, which designs DNNs based on iterative optimization algorithms, can give rise to hybrid model-based/datadriven receivers, presenting DetNet of [32] as an example. We identified that the main drawback of these aforementioned techniques in the context of digital communications stems from their usage of highly-parameterized DNNs applied in an end-to-end fashion, which directly results in the need for massive data sets for training. Then, we presented DNN-aided algorithms, where DNNs are integrated into existing model-based algorithms. We identified the latter as being extremely suitable for digital communications due to the wide variety of model-based algorithms designed for such setups, combined with its ability to incorporate different levels of domain knowledge as well as utilize compact DNNs as intermediate components in the inference process.
The DNN-aided symbol detectors presented as examples here, i.e., ViterbiNet [11], BCJRNet [12], and DeepSIC [43], all numerically demonstrated improved performance over competing strategies when a limited amount of training data is available. In particular, it is demonstrated that these DNN-aided symbol detectors, which are designed to operate in a hybrid model-based/data-driven fashion, learn to approach the performance achieved by purely model-based approaches operating with perfect knowledge of the underlying channel model and its parameters. Furthermore, the DNN-aided symbol detectors were shown to be notably more resilient to CSI uncertainty compared to model-based schemes, carrying out accurate detection in the presence of inaccurate CSI. Finally, it was demonstrated that model-based deep learning enables DNN-aided receivers to learn their mapping from relatively small data sets, making it an attractive approach to combine with tracking of dynamic channel conditions.