{'fraction_non_alphanumeric': 0.04530731395209341, 'fraction_numerical': 0.010237133602591274, 'mean_word_length': 4.095354523227384, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Extreme Learning Machine Based Non-Iterative and Iterative Nonlineraity Mitigation for LED Communications

This work concerns receiver design for light emitting diode (LED) communications where the LED nonlinearity can severely degrade the performance of communications. We propose extreme learning machine (ELM) based non-iterative receivers and iterative receivers to effectively handle the LED nonlinearity and memory effects. For the iterative receiver design, we also develop a data-aided receiver, where data is used as virtual training sequence in ELM training. It is shown that the ELM based receivers significantly outperform conventional polynomial based receivers; iterative receivers can achieve huge performance gain compared to non-iterative receivers; and the data-aided receiver can reduce training overhead considerably. This work can also be extended to radio frequency communications, e.g., to deal with the nonlinearity of power amplifiers.

eyes. At the receiver side, photo diodes (PDs) are used to detect the light intensity and convert it back to electrical signals. However, LED exhibits nonlinearity, which can significantly degrade the communication performance, in particular, when high order modulations such as pulse amplitude modulation (PAM) are employed to achieve high speed transmission [7], [8]. The nonlinear behavior of LED imposes amplitude distortion on the input signals. The lower peaks of signals are clipped at the LED turn-on voltage (TOV), and the upper peaks are saturated at the maximum allowable voltage recommended by the manufacturer [8].
A variety of nonlinearity mitigation techniques have been developed in the literature, which can be divided into two categories: pre-distortion at transmitter and post-distortion at receiver. The pre-distortion techniques require additional feedback link from receiver to transmitter or assume that the LED nonlinearity is known and fixed. The LED nonlinearity without memory effects was considered in [7] and [9], where a polynomial-based pre-distorter was proposed to compensate the LED nonlinearity. However, LED exhibits not only instantaneous nonlinearity, but also memory effect in high speed communications. A Volterra series based equalizer was proposed in [10] to mitigate LED nonlinearity with memory effects. However, the determination of Volterra coefficients requires high computational cost. To circumvent this issue, a memory polynomial based technique was proposed to postdistort the LED nonlinearity at the receiver [11]. However, these polynomial based methods involve the determination of polynomial coefficients with a matrix inverse operation, which can easily lead to numerical instability due to the ill condition of the matrix [12]- [14]. Meanwhile, as a nonlinear system may not be thoroughly compensated by a finite order polynomial, errors are inevitably incurred [15]. As postdistortion is deployed at the receiver side, it is more convenient to implement and is more suitable to handle the time-varying LED nonlinearity and VLC channel. Hence, we focus on postdistortion in this paper.
In this paper, we first propose an extreme learning machine (ELM) based non-iterative receiver to jointly mitigate LED nonlineraity and memory effects due to LED and VLC channel. Then, we consider iterative nonlinearity mitigation and decoding in a coded LED communication system. We design iterative receivers, which consists of an ELM based soft-in-soft-out (SISO) post-distorter and a SISO decoder. The SISO post-distorter and the decoder work in an iterative manner by exchanging log-likelihood ratios (LLRs) of coded bits. The ELM based post-distorter computes the extrinsic LLRs of coded bits based on the feedback from the SISO decoder and observations where ELM is used to handle both LED nonlinearity and memory effects. The ELM in the postdistorter can be trained with training sequence. To improve the transmission efficiency by reducing the training overhead, we further investigate a data-aided ELM based post-distorter, where both training sequence and data are utilized to train the ELM. We show that the training overhead can be significantly reduced in the data-aided ELM based scheme. It is shown that compared with the conventional polynomial based techniques, ELM is much more effective to combat the LED nonlinearity, which is demonstrated in Section V. It is noted that this work can be extended to RF communications, e.g., to handle the nonlinearity of power amplifiers. The rest of the paper is organized as follows. Section II gives a brief review of the ELM. Section III elaborates the signal model and details the design of ELM based non-iterative and iterative receivers for LED nonlinearity mitigation and decoding. In Section IV, a data-aided ELM based iterative receiver is designed and discussed. In Section V, experimental results are provided to verify the effectiveness of the proposed receivers. Conclusions are drawn in Section VI.

II. EXTREME LEARNING MACHINE
As shown in Fig. 1, ELM is a single-hidden layer feedforward neural network, where the hidden nodes (i.e., the input weights {ω ij } and biases {b i }) are randomly initialized and fixed without tuning [16]. The parameters to be learned in ELM are the output weights, and hence ELM can be formulated as a linear model with respect to the parameters, which boils down to solving a linear system, making ELM efficient in learning.
. . , s j(U −1) , s jU ] T and y j = [y j1 , y j2 , . . . , y j(Q−1) , y jQ ] T , the output of the SLFN shown in Fig. 1 can be expressed as where L is the number of hidden nodes, ω i = [ω i1 , ω i2 , . . . , ω iU ] T is the input weight vector that connects all input nodes to the ith hidden node, b i is the bias of the ith hidden node, and g(.) is the activation function of the hidden layer. β i = [β i1 , β i2 , . . . , β iQ ] T denotes the output weight vector connecting the ith hidden node and the output nodes. The output nodes are chosen to be linear in this network. By incorporating the N equations in (1), it can be represented compactly as where and H denotes the hidden layer output matrix. ELM randomly selects input weights and biases, and output weights β are obtained by minimize the cost function N j=1 ψ j − y j 2 , which is given by the least-squares (LS) where H † is the Moore-Penrose generalized inverse of matrix

III. ELM-BASED NON-ITERATIVE AND ITERATIVE RECEIVER DESIGN A. Signal Model
We consider a single-carrier coded LED communication system with PAM, which is shown in Fig. 2 for a system with a non-iterative receiver and Fig. 4 with an iterative receiver. At the transmitter side, the information bits are first encoded into a code sequence (and permuted by an interleaver in an iterative LED system as shown in Fig. 4). The encoded (or interleaved) code sequence c is split into a number of length-P subsequences {c n = [c n,1 , c n,2 , . . . , c n,P ]}, and each subsequence c n is mapped to a symbol x n ∈ D with a mapping rule (e.g., Gray mapping), where D = {α i , i = 1, 2, . . . , 2 P } denotes a 2 P -ary PAM symbol alphabet. Symbol x n is used to modulate the LED light intensity for transmission. As a result of the LED nonlinearity and memory effect, the transmitted signal is expressed as where f (.) represents the LED nonlinear distortion with memory effect and M is the memory length. The memory polynomial model is widely used in the literature, and in this case the LED nonlinearity with memory effect can be modelled as [8] where K is the order of the memory polynomial, and {a k,m } are the coefficients of the polynomial. It is noted that the memory effect of VLC channel, which causes inter-symbol interference (ISI), can be absorbed into the memory polynomial model. The signal is captured by the PD and converted to an electrical signal. We assume a unit PD responsivity for simplicity, and the received signal can be represented as where w n is the additive white Gaussian noise (AWGN) with zero mean and variance σ 2 .

B. ELM Based Non-iterative Receiver
We first introduce the conventional polynomial based noniterative receiver. For an LED system with nonlinearity and memory effects, memory polynomial based post-distorter has been investigated [11]. With the received signal {y n } as input, the output of the post-distorter can be represented as [11], whereK is the order of the memory polynomial used in the post-distorter,M is the memory length, and {a * k,m } are the coefficients of the polynomial. We can rewrite (8) in a vector form asx where By minimizing the cost function N n=1 x n − r T a * 2 , the LS solution to the coefficient vector a * can be calculated as [11], is the training sequence, and

Input Output
is constructed based on the received sequence {y n }. The coefficient vector a * can also be calculated recursively by using the recursive least squares (RLS) [11]. It has been shown that the matrix inverse in (12) suffers from numerical instability as R T R is usually an ill-conditioned matrix [12]. It is also noted that to enable the polynomial based post-distorter to have a high capability of LED nonlinearity compensation, a high order polynomial is required. However, a higher order polynomial can more easily lead to ill condition of the matrix R T R. This can result in significant performance degradation [13], [14]. In this work, we first propose an ELM based non-iterative receiver for the mitigation of LED nonlinearity and memory effects. The system diagram is shown in Fig. 2, where ELM works as a post-distorter to compensate the LED nonlinearity and memory effects. Here, we use a sliding window approach, i.e., the input of the ELM is a vector y n = [y n , y n−1 , . . . , y n−M ] T , which is obtained by windowing the received signal sequence, and the expected output of ELM is a single symbol x n in training. This is shown in Fig. 3, where the input and output dimensions of ELM are M + 1 and 1, respectively. Once the ELM postdistorter is trained using (4) and the trained output weights T is obtained, it can be used to predict the transmitted symbols {x n } based on the received signal {y n }, i.e., where y n = [y n , y n−1 , . . . , y n−M ] T . It turns out that ELM based post-distorter is much more powerful than the conventional memory polynomial based post-distorter in dealing with LED nonlinearity. The computation of the memory polynomial coefficients can easily suffer from numerical instability even for moderate order of the memory polynomial, which can severely impact the performance of memory polynomial based post-distorter. In contrast, the ELM based post-distorter works very well as demonstrated in Section V.

C. ELM Based Iterative Receiver
It is well known that iterative receivers are powerful to combat ISI in wireless communications [17]- [19]. Inspired by this, we also propose an ELM based iterative receiver for the mitigation of LED nonlinearity and memory effects. The iterative receiver is shown in the lower part of Fig. 4, which is composed of a SISO post-distorter and a SISO decoder, and they work in an iterative manner by exchanging the extrinsic LLRs of coded bits. The SISO post-distorter calculates the extrinsic LLRs for each coded bit with the extrinsic LLRs from the decoder as the a priori information. Then, with the extrinsic LLRs from the post-distorter, the decoder refines the LLRs with the code constraints. In this work, standard SISO decoding algorithms (e.g., the BCJR algorithm for convolutional codes) are employed and we focus on the design of the SISO post-distorter. The aim of the postdistorter is to calculate the LLR for each code bit c n,p , which can be represented as LLR p (c n,p ) = ln p(c n,p = 0|y) p(c n,p = 1|y) = ln αi∈D 0 p p(x n = α i |y) where D 0 p and D 1 p denote the subsets of all α i ∈ D whose label in position p has the value of 0 and 1, respectively. y = [y 1 , y 2 , . . . , y N ] is the received signal vector. According to the turbo principle [19], extrinsic LLR for each coded bit is passed to the decoder, which can be expressed as where LLR a (c n,p ) is the a priori LLR from the decoder in the previous iteration. It can be found from (17) that the key of the post-distorter is to compute the a posteriori probability of each transmitted symbol p(x n = α i |y). Based on the Bayes theorem, we have where p(x n ) is the a priori probability of symbol x n , and it can be computed based on the feedback from the decoder, i.e., where each α i corresponds to a binary vector Ω i = [Ω i,1 , Ω i,2 , . . . , Ω i,L ]. Next, we need to compute the likelihood p(y|x n ) for each x n . We use a sliding window approach to approximate the likelihood. Define y n = [y n , y n+1 , . . . , y n+M ] T and w n = [w n , w n+1 , . . . , w n+M ] T . We compute p(y n |x n ) instead of p(y|x n ). Based on (6) and (7), we have where 0 < q < M , a(x n ) is the useful signal related to x n , and i n is the interference with respect to x n .
We approximate the interference i n to be Gaussian with mean vector i n and covariance matrix V in . Then, based on (21), we have where m = a(α i ) + i n , and V is the covariance matrix of interference i n plus noise w n in (7), i.e., V = V in + σ 2 I.
To obtain the likelihood in (22), we need to compute a(α i ), i n and V . It can be seen that all of them depend on the LED nonlinearity, i.e., the memory polynomial coefficients {a k,m }. In [20], the iterative receiver is designed with the perfect knowledge of of LED nonlineraity and memory effects, i.e., assuming that the memory polynomial coefficients {a k,m } are perfectly known. However, they are usually unknown in practice. A possible solution is to first estimate the polynomial coefficients {a k,m } in (6) and then calculate a(α i ), i n and V . However, it can be difficult since the estimation of memory polynomial coefficients involves the inversion of a correlation matrix with ill condition, which can easily lead to numerical instabilities especially for high order or even moderate order polynomials [13], [21]. In addition, it can also be difficult to determine the polynomial order and memory length to properly model the LED nonlinearity and memory effects.
In this work, we use ELM to efficiently model the LED nonlinearity. The ELM is trained with a training sequence, and then a(α i ), i n and V can be estimated based on the trained ELM, which are detailed in the following. 1) ELM training: As we are using sliding window approach, given an input vector x n = [x n−M , . . . , x n−1 , x n , x n+1 , . . . , x n+M ] T , we use ELM to predict the corresponding output of the LED y n = [y n , y n+1 , . . . , y n+M ] T . Therefore, the dimensions of ELM input and output are 2M + 1 and M + 1, respectively, as shown in Fig. 5. We can use the training sequence {x n } and the corresponding received signals {y n } to train the ELM, and the output weights β * can be obtained based on (4). The trained ELM models the LED nonlinearity and memory effects, i.e., with the input vector x n = [x n−M , . . . , x n−1 , x n , x n+1 , . . . , x n+M ] T , the output of the ELM is approximately a(x n ) + i n , where a(x n ) and i n are defined in (21). This enables us to estimate a(α i ), i n and V to evaluate the likelihood p(y n |x n ) in (22), which is detailed in the following.
2) Estimate a(α i ) based on the trained ELM: Note that the input of the trained ELM is a vector x n , and the interference i n is determined by {x j , j = n}. Then a(α i ) can be estimated by setting x n = α i and x j = 0, j = n. Hence, with a special input vector x n = [0, . . . , 0, α i , 0, . . . , 0] T , the output of the ELM can be used as an estimate of a(α i ), which is illustrated in Fig. 6a.
3) Estimate i n based on the trained ELM: We model the interference i n as whereî n is an estimate of i n , and w in denotes a zero mean noise. Hence,ī n =î n . Next, we use the trained ELM to find the estimate of i n . As we know the interference i n is determined by x j , j = n in the vector x n , in order to getî n , we need the estimate of x j , j = n, which is denoted byx j . As we know in a turbo system,x j can be calculated based on the feedback from the decoder, which is the mean of x j [22], i.e.,x where p(x j = α i ) can be calculated based on extrinsic LLR obtained at each iteration. Therefore,î n can be obtained based on the trained ELM with a special input x n = [x n−M , · · · ,x n−1 , 0,x n+1 , · · · ,x n+M ] T , where we set x n = 0. This is shown in Fig. 6b.

Input Output
Estimate of .($ % ) (a) Estimate a(α i ) based on the trained ELM.

Input Output
Estimate of ̅ i #

4) Estimate V based on the trained ELM:
Recall that V defined in (16) is the covariance matrix of interference i n plus noise w n . It can be estimated aŝ However a(x n ) is unavailable, because x n is unknown. We note that the estimate of x n , i.e.,x n , can be calculated from the feedback from the decoder, so a practical estimator forV isV In addition, to reduce the complexity in computing (22), we further approximateV as a diagonal matrix by ignoring its off-diagonal elements. Hence, the matrix inverseV −1 is trivial and the evaluation of (22) actually only involves scalar operations, thus leading to low complexity. Noting thatī n is already obtained in the above, we only need to calculate a(x n ). This can be easily achieved by using the trained ELM with the input vector x n = [0, . . . , 0,x n , 0, . . . , 0] T , which is illustrated in Fig. 6c.

IV. DATA-AIDED ELM BASED ITERATIVE RECEIVER
We need sufficient training samples to train the ELM so that it can accurately model the LED nonlinearity and memory effects. By taking advantage of the iterative receiver, the estimated data based on the feedback from the decoder can be exploited as virtual training sequence for ELM training, i.e., ELM can be trained with both training sequence and data. This can significantly reduce the length of training sequence, thereby improving transmission efficiency. Different from Section III-C, where only the training sequence (it is exactly known) is used, we have to consider the uncertainty of the virtual training sequence because they are estimated and refined through the iterative process at the receiver. In this case, the estimation of ELM output weight β is based on the following model, where ∆H accounts for the hidden perturbation matrix due to the uncertainty of the virtual training sequence, ∆Y denotes unknown error between the observed signal Y and the true signal Y t . The total least squares (TLS) can be used for fitting such a model to data that minimizes errors in both the transmitted sequence and observed sequence [24]. The problem is now to find the smallest perturbations [∆H ∆Y ] to the measured signals that satisfy the model in (29), i.e., (30) Singular value decomposition (SVD) can be used to find the unique solution to the TLS problem in (30) [24], [25]. Take the SVD of the matrix [H Y ], i.e., Define the partitioning of matrix and Θ as .
The TLS solution exists if and only if θ 22 is non-singular, and the output weights β * are given by Note that the proposed data-aided ELM based iterative receiver consists of two phases: an initialization phase and a data-aided phase. In the initialization phase, the proposed nondata aided receiver in the Subsection III-C is used to estimate the nonlinearity merely using the training sequence due to the lack of feedback from the decoder. Following the initialization phase, the data-aided training phase commences and both the training sequence and the estimate of the data sequence calculated based on the feedback from the decoder are used to re-train the ELM to improve the modelling accuracy for the LED nonlinearity and memory effects.

V. EXPERIMENTAL RESULTS
In this section, we present experimental results to demonstrate the effectiveness of the proposed non-iterative and iterative ELM based receivers in handling LED nonlinearity and memory effects. A convolutional code with generator [171, 133] is used for the coding scheme, and the APP decoder is implemented using the BCJR algorithm [26]. A high order modulation 8-PAM is used. We consider a commercial LED (Kingbright blue T-1 3/4 (5mm) LED) whose I-V curve (extracted from the datasheet [23]) is shown in Fig 7. The Hammerstein model is employed to include both LED nonlinearity and memory effects in the simulations [27] (Noting that other models can also be used to generate the distorted signal). The Hammerstein model is composed of a nonlinear block followed by a linear time-invariant (LTI) block, which is expressed as where ρ 1 = 0.15 and ρ 2 = 0.05. AWGN is added at the receiver side. The signal-to-noise ratio (SNR) is defined as E(z 2 n )/σ 2 n , where σ 2 n is the variance of the noise. We use sin as the activation function in the hidden layer for ELM and its input weights and biases are randomly distributed from We first examine the performance of non-iterative receivers in Section III-B with a training length of 800. The conventional RLS based post-distorter [11] is implemented with the order K = 7 and the memory lengthM = 4. The ELM has 100 hidden nodes in the ELM based non-iterative receiver. The BER performance is shown in Fig. 8. We can see that the RLS based post-distorter simply does not work. In contrast, the non-iterative ELM based receiver works very well. The conventional polynomial based method can easily suffer from numerical instability, thereby leading to poor performance. However, the neural network-based method does not have such issue. From Fig. 8, we can see that neural network-based method is much more effective in combating the nonlinearity. Figure 9 shows the BER performance of the LED system with various receivers. The performance of the iterative re- ceiver with the perfect knowledge of the nonlinearity and memory, is also included as a performance bound. The training sequence length is 800 except the 'Proposed 1200' (which is detailed later). As shown in the Fig. 9, the RLS based post-distorter [11] does not work properly. Fig. 9 also shows the performance of the receiver which employs the frequency domain equalization (FDE) to compensate the memory effect and the conventional polynomial inverse technique to mitigate the nonlinear distortion of LED (denoted by 'FDE+InvPoly') [28], where nonlinearity and memory are assumed to be perfectly known. It can be seen that the receiver does not work well. In contrast, the proposed non-data-aided ELM based iterative receiver (with 150 hidden nodes) delivers much better performance. Fig. 10 shows the BER performance of the proposed non-data-aided ELM based iterative receiver with different iterations, which demonstrates that the converge of the iterative receiver is fast, i.e., only 4-5 iterations are needed.
To demonstrate that the capability of the proposed ELM based iterative receiver can be significantly improved with longer training sequence and more hidden nodes, we increase the number of hidden nodes to 300 and the length of the training sequence is increased to 1200. Its performance is also shown in Fig. 9, where it is denoted by 'Proposed 1200'. We can see that the performance of the proposed receiver is improved further and it can approach the performance bound closely at relatively high SNRs. However, it requires high training overhead. This can be overcome by using the proposed data-aided scheme where both training sequence and data are employed to train the ELM. Fig. 11 shows the BER performance of both non-dataaided ELM based iterative receivers and data-aided ones with different training lengths. It can be observed that the dataaided ELM receiver with 400 training symbols and 400 data symbols outperforms the non-data-aided ELM receiver with only 400 training symbols, significantly. We can also see that the performance of the data aided receiver with 400 training and 400 data symbols can approach that of the non-data-aided ELM receiver with 800 training symbols at relatively high SNRs, i.e., training overhead is reduced by half in the dataaided ELM receiver.

VI. CONCLUSIONS
In this paper, we have proposed novel ELM based noniterative and iterative receivers to jointly handle LED nonlinearity and memory effects. It is shown that the ELM based non-iterative receivers outperform the conventional receivers significantly, i.e., neural networks are much more efficient to deal with LED nonlinearity compared to the conventional polynomial techniques. For the iterative case, we have designed non-data aided and data-aided receivers by taking advantage of the iterative process in an iterative receiver, where data can serve as a virtual training sequence. It has been shown that a huge performance gain can be achieved by iterative receivers compared to non-iterative receivers, and the data-aided receiver can reduce considerable training overhead compared to the non-data aided receiver. It is worth mentioning that this work can also be extended to RF communications.