{'fraction_non_alphanumeric': 0.033782984262217486, 'fraction_numerical': 0.011004614838480652, 'mean_word_length': 4.551067323481116, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Machine learning for decoding linear block codes: case of multi-class logistic regression model

Facing the challenge of enormous data sets variety, several machine learning-based algorithms for prediction (e.g, Support vector machine, multi layer perceptron and logistic regression) have been highly proposed and used over the last years in many fields. Error correcting codes (ECCs) are extensively used in practice to protect data against damaged data storage systems and against random errors due to noise effects. In this paper, we will use machine learning methods, especially multi-class logistic regression combined with the famous syndrome decoding algorithm. The main idea behind our decoding method which we call logistic regression decoder (LRDec) is to use the efficient multi-class logistic regression models to find errors from syndromes in linear codes such as bose, ray-chaudhuri and hocquenghem (BCH), and the quadratic residue (QR). Obtained results of the proposed decoder have a significant benefit in terms of bit error rate (BER) for random binary codes. The comparison of our decoder with many competitors proves its power. The proposed decoder has reached a success percentage of 100% for correctable errors in the studied codes.


INTRODUCTION
In the last few years, telecommunication technologies have known huge innovations in order to have good speed, and reliable communication between all connected objects, e.g, smartphones, computers, and cameras. The internet of things (IoT) combined to the arrival of 5G in the 3 rd generation partnership project [1] have guaranteed very fast speed of data transmission, compared to the older version 4G long term evolution (LTE) [2], the new radio (NR) for 5G adopts a new strategy in error-correction by using transformation in the data channel which uses low density parity check codes (LDPC) and the control channel which uses 2 dimensional tail-biting convolutional codes (TBC) [3]. This technology invests in the hardware and transmission's power to reduce the effect of signal noise. In the same context, some researchers invest in software development, Giuseppe Aceto [4] has conducted a study by using deep learning (DL) for classifying the mobile encrypted traffic. A novel approach has been presented by Tim O'Shea [5] in the design of communication by using DL for the physical layer. In our case, the design of our model is based on the simplified model of communication system, combined with the logistic regression classification in the process of decoding channel Figure 1. Depending on the way used to add redundancy in the messages to be transmitted, it is possible to divide the error correcting codes into two major families: block codes and convolution codes [6]- [7]. The block codes encode data by block independently of other blocks whereas the convolutional codes process the current block not only in relation to the current input of the channel encoder but also in relation to the previous entry blocks [8]. In Figure 2, we present a classification of linear block codes. The category of linear-block codes is described as a code C(n,k,d); its length is n; its dimension is k, and its minimal distance is d. The particularity of this type of codes is that each linear combination of codewords is another codeword. In this paper, we will work on block codes, namely the category of systematic binary linear block codes. A linear block code C(n, k) is a set comprising 2 k code words so that the linear combinations of the k information bits would generate n bits of each codeword. This property allows to define C by a matrix G called generator matrix or coding matrix which can be used to associate with any information word m = (m 1 , m 2 ,. . . , m k ) composed of k bits a codeword c = (c 1 , c 2 ,. . . , c n ) as illustrated in the following Figure 3. The matrix G can be written in systematic form as in (1) where I k is the identity matrix of order k and P is a binary matrix of order (k,n-k). In this respect, for any code C(n, k), we can calculate a matrix H of order (n-k, n) whose lines are orthogonal to the lines of G, that is to say G * H T = 0. The matrix H is called the Ì ISSN: 2502-4752 parity matrix (control matrix) of the code C, and the matrix G the generator matrix. The vector space C ⊥ is a linear code called the dual code C of dimension n-k and of length n. If the code C is generated by systematic generating matrix G, its dual code C ⊥ can be generated by the matrix H as follows: The BCH code for communication and storage data has been discovered by Hocquenghem [6]. It was developed by Bose and Ray-Chaudhuri [7], it is one of most famous codes of powerful error capability [8]. The technical specification of BCH codes is as follows: In this paper, we will first expose a general literature review of methods and approaches in error correcting codes. Then, we will develop our new decoder LRDec based on logistic regression model which we think is one of the most efficient machine learning algorithm. We should inform you that we have combined our LRDec with the syndrome decoding technique so as to eliminate the decoding syndrome motive table. After that, we will expose different obtained results for linear codes. Finally, we will compare and discuss them with other existing decoders' results (e,g: HSDEC [9]- [10], ARdecGA [11] and BERT [12]). In addition to this, in contrast to other decoders, our LRDec is has been successfully applied to another kind of linear codes which is named quadratic residual code (QR). Hence, the comparison between our LRdec and HSdec in QR codes has shown a significant efficiency in terms of BER. The sequel of this paper is organised as follows: In section 1, we will present some related works. In section 2, an overview of machine learning techniques, especially the logistic regression models method will be detailed, and we will develop the proposed framework of the LRDec. In section 3, we will expose our results and discussion. At the end of this paper, we will present our conclusion and suggest some possible future directions of this work.

2.
RELATED WORKS Being aware of the difficulties of ECC problem, many decoders are used to enhance and improve the reliability and performance in terms of bit error rate (BER). Figure 4 presents the main classes of decoding techniques. Some decoders are based on algebraic theory such as the algorithms developed through solving nonlinear multivariate equations obtained from the identities of Newton [13]- [14], the Berlekamp-Massey algorithm [15] which is based on the calculation of syndromes and the definition of an error locator polynomial, the algorithm of Chase [16] and the algorithm of Hartmann Rudolf [17]- [18]. However, some algebraic techniques, aforementioned, require a large wide of computational operations, in terms of sum and product, in the used finite field. This time complexity makes their implementation in real time systems very hard. That is why, the algorithms with fast detection and correction are strictly required here. To solve this problem, several researches have been carried out to develop both heuristic or meta-heuristic algorithms and machine learning techniques which aim to detect and correct transmission occurred errors with high accuracy and high speed.
There are other methods that use non-algebraic techniques, such as genetic algorithms that belong to evolutionary techniques [18]- [19]. We have also found a work that uses local search to find errors [20]- [21]. Moreover, several other works use hashing techniques [22]- [23]. In this regard, the decoder named HSDec [9] based on syndrome calculation and hash techniques has significantly contributed to accelerate the search time of the error vector in the pattern error table. Also, we have found many works that deal with error correction by involving the deep learning algorithms [24]. The authors of these works exploit the properties of linear codes with the functionalities of deep learning algorithms to develop a decoder of BCH codes. Another decoder, applicable to polar codes, based on deep neural networks has proved to be a very efficient polar decoder [25]. A deep learning algorithm to ameliorate and improve the belief propagation algorithm [26] is applicable to BCH codes. The researchers exploited a machine learning algorithm combined with the syndrome calculation to improve the decoder performance in terms of BER and time complexity; this method is applicable to linear block codes [27].

THE PROPOSED MODEL
In this section, we will firstly give a brief summary of the techniques of machine learning. Secondly, we will present the design of our decoder LRDec as well as the technical specifications of its implementation.

Machine learning models
Machine learning is an extensively employed domain in artificial intelligence that is working on analyzing and exploring features of any kind of data, including nominal data, with the purpose to generate models and make an accurate future prediction. The latter can be presented by the way of functions construction from the data X=(X 0 , X 1 , X 2 , .., X j , ..) to predict the values of Y. This is illustrated in Figure 5. If the type of Y is discrete values, we talk about classification learning. But, if the type of Y is a continuous numeric value, the learning is regression type. For the classification problem, there is a set of algorithms in machine learning, such as K-nearest neighbors (KNN) algorithm,support vector machine (SVM) algorithm and logistic regression (LR) algorithm that can be utilized to produce very high classification accuracies. The most prevalent approach for evaluating binary response data is logistic regression. The logistic regression model estimates the likelihood of output Y as a function of one or more predictor X j . Despite its name, the logistic regression model is a classification model, rather than a regression model. It is a simple and powerful method to solve problems with binary and linear classification. In this paper, we will use the logistic regression, and namely, the multi-class logistic regression model.
In a classification problem, the objective is that the probability of the correct class Y to which the inputs x i belong must be maximized. We train the logistic regression model to find the appropriate weights, which will enable us to calculate the probability of each input belonging to the class Y. The Sigmoid function will always be used in this case. All these specifications are presented in Figure 6.
In the same way, as shown in Figure 7, we calculate the values f (z i )/i = 1, . . . , k, for k classes. Then, we compare these values with each other to find the target class. The values should be normalised so as to be considered as probability [28]. For this purpose, the softmax function is used.

The proposed decoding method based on logistic regression model
In this section, we will present our proposed model named LRDec for linear bloc codes. In this regard, we will work on BCH and Quadratic Residue codes. This decoder can be generalized to any linear block code defined by its generator matrix or its generator polynomial. This new LRDec based on the logistic regression will be compared to many famous decoders (HSDEC, ARDecGA and BERT) in terms of BER performance and complexity. LRDec is designed to improve syndrome decoding technique. In Figure 8 we present the architecture of the LRDec. On the one side, The features inputs S i are the n-k bits of the received word's syndrome. On the other side, the outputs Y i are the classes of all the different correctable errors converted to decimal values. Before any model creation procedure, i.e. the training phase, it is essential to prepare the input data as well as the output data. For the output, the aim is to define specific classes for each correctable error. As for the input, the objective is to generate all possible syndrome vectors with length equal to n-k.

ISSN: 2502-4752
Ì 543 a-Outputs Y We start by generating all possible errors of length n presented by the binary vector with weights (number of 1 in each vector) less than or equal to the error correcting capability of the code. After that, we have to convert all the generated vectors to decimal values. Finally, we group all these decimal values to list Y 1 . This list represents the classes for all correctable errors. As for the incorrigible errors, we are going to add Y 2 = 0, to keep the received word when it 's impossible to correct.
b-Inputs X After having created the list of classes Y with Y 1 for the correctable errors and Y 2 for incorrigible errors, we will create another list X by generating all syndromes of errors. After this process we will assign labels for each x i in X. The correctable errors Binary(y i ) must have a label x i = syndrome(ConvertBin(y i )); and for the other x j the corresponding label is Y 2 =0.
In general, the database will be formed by p = 2 n−k samples and m = t i=0 C n i classes: Syndrome of correctable error 1 Syndrome of correctable error 2 . . .

Syndrome of correctable error m Syndrome of incorrigible error m+1
. . .

Syndrome of incorrigible error p
Example: BCH (7,4,3) In this code, the first step is to generate all the possible correctable errors Err i , the second step is to calculate all the possible syndromes x i by calculating the product between errors and the matrix of the parity check H for BCH (7,4,3). In Table 1, we show the results of the generating data-set for the BCH (7, 4, 3) code. In Table 2, we show the number of classes Card(Y)= t i=0 C i n , andsamplesCard(X)=2 n−k for creating LRDec model.

The training process
Once the data-set (X; Y) is prepared, the next process is training the Logistic regression model, knowing that the machine learning paradigm discussion between over-fitting and under-fitting is not necessarily included in our strategy because the data-set presents all possible values. In this case, having a training error equal to 100% does not mean that we have the over-fitting case. However, it is our main objective to have a model which corrects all possible errors of weights ≤ t. In Table 3, we show that our decoder correct all errors of weight less than or equal to the error correcting capability of the studied codes. Thus the percentage success of LRDec on these codes is 100% for correctable errors.

RESULTS AND DISCUSSION
In order to show the huge success of our LRDec, we are going to plot its performances (for some linear BCH codes) in terms of bit error rate (BER) for different values of signal noise ratio (SNR), in the additive white gaussian noise channel (AWGN) and with binary phase shift keying (BPSK) modulation. The simulation parameters are displayed in the following Table 4:

Comparison and discussion
To show the efficiency of our decoding model, we have compared its performances with other existing decoders. In Figure 11, we present the performances of the LRDec, ARDecGA [11] and BERT [12] decoders for the BCH code (15,7,5). Thanks to the comparisons listed above, the LRDec for the code BCH (15,7,5) has proved to be the best decoder in terms of performance and efficiency. For the same purpose, as shown in Figure 12, the LRDec and the HSDec [9] have the same performance in the code BCH (31, 16,7). What is more, in Figure 13, we have compared the performance of LRDec with HSDec for a new class of binary code, the QR code (23,12,7), and the results have shown that the RLDec is more efficient than HSDec.

CONCLUSION
In this study, we have focused on the possibility of using a logistic regression model in error correcting codes, especially in syndrome decoding technique applied to linear codes. To achieve this objective, we have successfully designed a new decoder named LRDec with 100% accuracy of training model. The idea behind this choice of methodology is based on the performances guaranteed by machine learning algorithms. Unlike the classical syndrome decoding method, this new decoder LRDec does not need a large syndrome pattern's table. The proposed decoder LRDec has shown significant efficiency in term of BER and has reached the 100% correction of all correctable errors in the studied codes. We have also conducted a comparison between LRDec with three decoders: BERT, HSDec and ARDecGA for decoding BCH (15,7,5), BCH (31, 16,7), and QR (23,12,7) codes. The obtained results have shown the success of our proposed model decoder in learning how to calculate directly error from syndrome without using the large table of syndrome decoding process. In perspectives, we plan to study other machine learning models for decoding other non linear codes, and to search about the optimisation of the model by reducing its complexity in terms of the activation function or by discussing the model's parameters in order to improve its efficiency.