{'fraction_non_alphanumeric': 0.025251076040172165, 'fraction_numerical': 0.008672086720867209, 'mean_word_length': 5.21478105805429, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep learning-based discriminant model for wearable sensing gait pattern

In order to effectively improve the accuracy of identifying the gait pattern of wearable sensing data, this paper proposes a new model for deep learning gait mode discrimination that integrates convolutional neural network and long short-term memory neural network, which makes full use of the convolutional neural network to obtain the most local spatial characteristics of data and the long short-term memory neural network to obtain the inherent characteristics of the data, and effectively excavates the hidden high-dimensional, nonlinear, time-space gait characteristics of random wearable sensing timing gait data that are closely related to gait pattern changes, to improve the classification performance of gait mode. The effectiveness of the proposed model in this paper is evaluated using the HAR dataset from University of California UCI database. The experiment results showed that the proposed model in this paper can effectively obtain the time-space gait characteristics embedded in the wearable sensor gait data, and the classification accuracy can reach 91.45%, the precision rate 91.54%, and the recall rate 91.53%, and the classification performance is significantly better than that of the traditional machine learning model, which provides a new solution for accurately identifying the gait mode of wearable sensor data.


Introduction
In recent years, the construction of a machine learning gait classification model with superior generalization performance based on gait data obtained from outdoor environment has received wide attention in the field of gait pattern recognition research, which is of great significance for the prevention of falls in the elderly, the diagnosis and treatment and rehabilitation evaluation of elderly neurological functional diseases, human identity identification. It now has become a new research hotspot in the research field related to gait pattern recognition [1,2] . In recent years, with the rapid development of advanced data acquisition technology, some advanced data acquisition technologies (such as computer video, wireless radar, wearable sensors, etc.) have been used to collect gait pattern data in outdoor environments. For example, based on gait image data collected by computer video, some scholars have discussed the study of outdoor human gait pattern recognition in different perspectives [3] ; other scholars have discussed the research on gait pattern recognition in outdoor environment containing micro-Doppler feature information based on the gait data obtained by wireless radar devices [4] . While some scholars have also discussed the study on gait pattern recognition in outdoor environment based on gait data of wearable sensors (accelerometer, gyroscope, magnetometer, etc.) [5] . The studies found that the gait acquisition technology of cheap and portable wearable sensor has the advantages of adapting to different outdoor application scenarios and containing rich gait characteristic information, which can better avoid the loss of valuable gait characteristic information by computer video technology due to outdoor environment, human wearing clothing, and the loss of wireless gait detection signal because of the external environmental interference of wireless radar devices, which helps to improve the gait pattern recognition efficiency, and has been widely used in related research in recent years.
Based on wearable sensor data, the application of machine learning algorithms to explore gait pattern recognition models with superior generalization performance has received continuous attention from relevant research, and its basic idea is that it can make full use of the superior data learning performance of machine learning algorithms to obtain more representative gait characteristic information from wearable sensor gait data and improve the gait pattern recognition performance. In the early days, some studies explored the quantitative analysis of wearable sensor data based on traditional machine learning algorithms (such as decision trees, multi-layer perceptual neural networks, support vector machines, K-neighbors, etc.), and tried to construct gait pattern recognition performance with superior generalization performance [6,7] . For example, Bao et al. [8] discussed the gait pattern recognition model of ID3 decision tree based on the gait data of the triaxial accelerometer to identify three gait patterns such as normal walking, jogging, and stair climbing, with an average recognition rate that was only 79%. Tahafchi et al. discussed the application of KNN classification algorithm to obtain and compare data from wearable data of Parkinson's subjects (including triaxial acceleration data, gyroscope data, magnetometer data, and dual-channel non-invasive myoelectric scanner data), and the gait pattern recognition rate reached 91.9%, 87.1%, 80.9%, and 79.9% according to the participants, respectively [9] . In addition, based on the acceleration gait data of wearable sensors, Nickel et al. discussed the research on the construction of gait pattern recognition model based on support vector machine, invisible Markov model and KNN classification algorithm, among which the Equal Error Rate (EER) of support vector machine and invisible Markov model was 10.00% and 12.63%, respectively, and the Half Total Error Rate (HTER) of KNN classification algorithm can reach 8.24% [10,11] . The study found that the traditional machine learning algorithm has the advantages of low computational complexity in processing wearable sensor gait data to recognize the gait pattern, but because of its inherent linear computing model architecture, it is difficult to obtain more representative gait characteristic information hidden in the intrinsic structure of wearable sensor gait data, and it is difficult to support the construction of a gait pattern recognition model with superior generalization performance. In recent years, with the rapid development of emerging machine learning theories such as deep learning and the successful application of image processing, some scholars have tried to explore the construction of deep learning gait pattern recognition model based on the wearable sensor gait data, and its basic idea aims to make full use of the excellent data learning performance of deep learning algorithms to obtain more representative gait feature information from high-dimensional wearable sensing gait data and to improve the gait pattern recognition performance. For example, Zou et al. [12] based on the acceleration data and gyroscope data collected by wearable smartphones, the construction of convolutional neural network and recurrent neural network fused with gait pattern recognition model is discussed, try to obtain the inherent spatiotemporal correlation characteristics information of wearable sensing gait data to improve the performance of gait pattern recognition. The results showed that the accuracy of this method in pedestrian identification and identity authentication is found to be higher than 93.5% and 93.7% respectively. In addition, Ding et al. [13] proposed a gait pattern recognition model based on the long short-term memory algorithm LSTM based on wearable gait data (wearing an inertial measurement unit on the calf to collect angular velocity data), which aims to obtain the time-correlated gait feature information hidden in wearable gait data through the long short-term memory algorithm to detect the gait phase and use the phase marker data to train it. The experimental results show that the recognition accuracy rate can reach 91.4%. In recent years, although the research on gait pattern recognition based on deep learning has achieved good results and positive progress, there is still a lack of a technical means to accurately obtain the more representative time-space correlation gait characteristic information implied in wearable gait data, which seriously restricts the gait pattern recognition performance. Relevant medical studies have shown that gait is a walking posture of the human body, which is closely related to physiological factors such as human nervous system, motor system and psychological cognitive system, and it is a long-term memory process in which various physiological factors interact and influence each other, while the self-recurrent neural network model used in the current study only has short-term memory performance, and it is difficult to obtain long-term temporal correlation characteristic information during gait process. New deep learning models that acquire more representative spatiotemporal correlation gait feature information implied in wearable gait data are urgently needed.
Therefore, based on the wearable sensor gait data, this paper proposes a new model for deep learning discrimination of gait mode that integrates convolutional neural network model and long short-term memory neural network model, which aims to make full use of the convolutional neural network model's superior characteristics of obtain-ing the most representative feature information characteristics in local space of data and the inherent long-term time correlation characteristic information characteristics of long short-term memory neural network model, and accurately obtain the more representative spatiotemporal correlation gait feature information implicit in wearable gait data, improve gait pattern recognition performance. In addition, this paper selected the HAR data from the publicly available UCI database of the University of California, Irvine [14] , and compared with traditional machine learning algorithms and deep learning algorithm models to verify the effectiveness of the proposed model algorithms.

CNN-LSTM deep integration learning gait pattern discriminant model
The CNN-LSTM deep learning model proposed in this paper aims to make full use of the excellent characteristics of CNN and LSTM models to obtain the inherent spatial and temporal correlation characteristic information of data structures, respectively, and to deeply integrate the two from wearable sensing gait data (such as acceleration, gyroscope, etc.) to obtain more time-space correlation characteristic information that is closely related to gait changes, and improve the gait pattern recognition performance. That is, it is assumed that the gait pattern needs to be identified as the database = { 1 , 2 , . . . , }, among them, l represents the number of gait patterns to be recognized. Suppose the wearable sensor gait time series data is: In the series data, = ( 1 , … , ) is the wearable sensing data at time j, while m and t denote the number of wearable sensors and the number of gait pattern time series samples, respectively. In the research, each gait pattern time series data is selected, so that each data segment ℎ = ( −1 , ) contains gait spatiotemporal feature information, and all selected data segment are defined as dataset = {ℎ 1 , … , ℎ , … , ℎ }, and the k is the number of all selected data segment.
In order to accurately identify the gait pattern, we need to construct a model Γ to obtain the vector Yi containing the gait feature information from each data segment hi, that is, = ( , ℎ ). Then, the confidence value set P corresponding to each gait pattern vi is calculated based on an inference method Ψ, : ( / , ) = ( , ), is a training parameter set based on the model Γ. Then, by computing the following maximum score value: * = arg max ( / , ), the gait pattern * can be obtained accurately, and each gait pattern can be recognized. In this study, we constructed the CNN-LSTM deep fusion learning model as a model Γ, and first use the CNN deep learning model to obtain the local spatial feature information that is closely related to the gait pattern changes from each data segment hi. On this basis, based on the LSTM deep learning model, the temporal correlation of the local spatial features of the gait data is obtained, and more the time-space feature information related to the gait pattern changes can be obtained, the gait pattern * is obtained with the maximum probability, and the gait pattern is accurately identified.
The framework of the gait pattern discriminant model based on CNN-LSTM fusion deep learning proposed in the paper is shown in Figure 1, which consists of three parts: gait data input layer, CNNLSTM fusion deep learning, and full connection layer. As shown in Figure 1, in view of the time-space correlation characteristics of wearable gait sensing data, the CNN is composed of three convolutional layers (CL1, CL2, CL3), a maximum pooling layer (MP1), and two dropout layers, which accurately obtain the most representative local spatial features inherent in the gait data. In order to accurately obtain the temporal correlation of the most representative local spatial features in the gait data, the LSTM model consists of 32 cells, and in order to accurately obtain the temporal correlation of the most representative local spatial features in the gait data, the fully connected layer is consists of 6 cells, and the gait pattern is identified with the maximum probability.
(1) Extract the most representative local spatial characteristics of gait data based on CNN.
In order to effectively obtain gait feature information, the data of the wearable sensing gait time series at t time t is defined as: Among them, BA-XYZ represents the three-dimensional human motion acceleration data, GAXYZ represents three-dimensional gravitational acceleration data, and Gy-XYZ represents three-axis gyroscope data. For ease of analysis, selects ∈ {1, … ,128}, and its sensory gait data input sequence is defined as: Assuming that the CNN model used to obtain the most representative gait local spatial features has a convolutional layer, each layer of convolutional kernels is defined as: M1 × N1, the ∈ {1, … , } convolutional layer extracts the gait local spatial feature F (l) , which is defined as: Where f(‧) represents the activation function, ﹤‧﹥represents the inner product, and b (l) is the bias term; w (l) is a one-dimensional convolutional kernel vector; ø is the length of w (l) .
In view of the high-dimensionality, nonlinearity, randomness and low algorithmic complexity of the wearable sensing gait data defined in equation (3), this paper constructs a three-layer one-dimensional convolutional layer, each of which has 32 convolutional kernels, the size of which is defined as 3 × 3, the step size is defined as 1, and the ReLUfunction [15,16] with good nonlinear characteristics is used as the activation function. According to equation (3), the size of the wearable sensor gait input data is defined as 128 × 9, and the gait local characteristic data of the output by the first, second, and third convolutional layers can be obtained, respectively: 126 × 32, 124 × 32 and 122 × 32. To effectively maintain good learning performance and avoid overfitting, build a Dropout layer.
In order to effectively maintain the intrinsic characteristics of the gait features obtained by the convolutional layer, reduce its redundancy information, and use the pooling layer to reduce the characteristic dimensionality and increase its spatial invariance [17] , the pooling layer that defines the maximum pooling technology obtains the local spatial characteristics Pj that contains more gait change information, which is defined as: R represents the pooling window size.
Therefore, based on equation (5), the local spatial characteristics with the most gait change information can be obtained from the wearable sensor gait time series data, which lays the foundation for subsequent acquisition of its temporal correlation features. We used this local gait feature as an input to LSTM to extract the dependent characteristics of gait data for a long period.
(2) The temporal correlation of local features of gait data extracted based on LSTM layer A gait activity can be considered as a long series of time series, and the long-term time-dependent characteristics of local features can be effectively extracted by establishing an autoregressive model RNN. In this paper, in view of the good autoregressive network architecture characteristics LSTM with intrinsic time correlation of dynamic learning time series data [18] , the LSTM cell is constructed, including 1 memory cell C and 3 gate functions (input it, forgetting ft, output ot), and the intrinsic long-term time-related characteristics of gait data is extracted in real time, the specific implementation is as follow.
Assuming the gait data sample represented by p t is processed by the CNN model at the t moment as the input term of the LSTM neuron, and when passing through the cell of the LSTM, the useless extracted data information is first discarded by the forgetting gate, and its output is: Where σ represents the activation function Sigmoid, Wf is the weight, and bf is the bias value. The updated data information is then determined by input gate it and candidate memory cell ̃: The Wi and Wc refer to weights, and bi and bc refer to bias values. The cell update status of the LSTM is then represented by the memory cell Ct: Finally, the output data information ht of the LSTM unit is determined as: The ot is the output gate; ht is the output of the current neuron in time. Specific derivation equations can be referred to reference [19] . By retaining the information that has undergone forgetting and input through the above memory units, the LSTM unit can effectively transmit historical information with a long-time interval to obtain the intrinsic time correlation characteristics of the data. The LSTM layer proposed in this paper consists of 32 cells to process the time signals which expressed as one-dimensional eigenvectors as shown in equation (12).
The feature vectors s is processed by a fully connected layer composed of 6 cells, and the output is: W is the weight matrix of the fully connected layer; ɛ is the bias term vector. We set the activation function of the fully connected layer to the Softmax function, and the final output is: * = ∑ , ∈ {1, … ,6} The gait pattern vi is identified with maximum probability by the equation (14).
From the above analysis, it can be seen that the CNN-LSTM model proposed in this paper fully integrates the excellent the characteristics of both CNN and LSTM to obtain the most representative temporal and spatial gait features inherent in gait time series data, reduces the complexity of the learning network structure and the large training cost of the model, enhances the nonlinear fitting performance of the fusion deep learning algorithm, and helps to improve the accuracy and precise in the gait classification of the proposed model. The neural network model proposed in this paper uses the classification cross-entropy loss function to minimize the classification error rate of the training sample, which is defined as: D represents the training set, W represents the weight matrix, and B represents the bias value; N indicates the number of training samples, y (i) represents the label of the i th sample, and ŷ represents the predicted label and﹤‧﹥represents the inner product.

Experimental data acquisition
This paper uses the HAR dataset from UCI database for machine learning proposed by the University of California Irvine. The dataset collected 6 gait patterns from 30 volunteers aged 19 to 48: standing, sitting, lying down, walking, going upstairs and downstairs. Each subject performs two experiments.
Scheme: In the first experiment, the smartphone (with built-in accelerometer and gyroscope) was worn on the left side of the waist; in the second experiment, the subjects placed their smartphones randomly.

Data preprocessing
In order to effectively eliminate noise interference and obtain more useful gait data, we used a median filter and a third-order low-pass Butter-worth filter (cutoff frequency set to 0.3 Hz) to cancel the noise processing of human acceleration signals and gravitational acceleration signals. Set the window width to 2.56s for sliding window data, window overlap is set to 50%, that each window has: 2.56 s × 50 Hz = 128 cycles, and fast Fourier transform was used to obtain the 17 gait data time-domain and frequency-domain gait features. Therefore, 17 metrics were used to evaluate the eigenvectors in the time and frequency domains, a total of 561 features were extracted to describe each active window (sample point), each sample point is regarded as a gait mode, the metrics are shown in Table 1 below.

Selection of evaluation criteria for gait classification performance
In order to objectively and accurately evaluate the generalization performance of the gait classification model proposed in this paper, the classification accuracy, gait precision, and recall rate commonly used in gait classification related studies were selected as the objective evaluation indicators of gait classification performance.
(1) Accuracy: Used to objectively evaluate the accuracy of the gait deep learning classification model proposed in this paper, which is defined as: TP represents the number of samples that correctly identify gait patterns; FP represents the number of samples that incorrectly identify gait patterns; TN represents the number of samples in which the correct gait pattern was incorrectly recognized as another gait pattern; The FN represents the number of samples of which the gait pattern was incorrectly recognized as the correct gait pattern.
(2) Precision: It is used to objectively evaluate the performance of the gait deep learning classification model proposed in this paper to "truly" identify gait patterns, which is defined as = + (17) (3) Recall: It is used to objectively evaluate the performance of gait deep learning classification models proposed in this paper for correct recognition of gait patterns, which is defined as

Experiment results
The experiments in this paper are based on Google's open source deep learning framework Tensorflow, the specific experimental platforms are CPU(i5), Python3.7, Keras2.3, and Tensorflow2.1. The number of samples was 10 299, 70% was randomly selected as the training set, while 30% was the test set, and the experimental data is sent to the model training in batches, and the batch size is 32 data samples. The number of training rounds of the model is set to 30 and the adaptive learning rate optimization algorithm Adam is used and the learning rate is set to 0.001.
(1) Optimal structural parameter selection of gait deep learning classification model In order to accurately optimize and design the structure of the gait deep learning classification model proposed in this paper and improve its performance, this paper first quantitatively evaluates the number of convolutional layers and the number of neurons in the long short-term memory network selected by the proposed model optimization. The selection result of the convolutional neural network convolutional layer is shown in Figure 2. From Figure 2, when the number of convolutional layers increases from 1 to 3, the accuracy of the model gradually increases. When the number of convolutional layers is 3, the classification accuracy is the largest which up to 92.1%. But when the number of convolutional layers increase to 4 and 5, the classification accuracy decreases significantly. The results show that when the number of convolutional layers is 3, the gait fusion deep learning classification model proposed in this paper can obtain more gait characteristic information closely related to gait pattern changes through the wearable sensing acceleration and gyroscope gait data, which effectively improves the classification performance of the model. However, when the number of convolutional layers increases to 4 and 5, it is difficult to obtain some representative feature information from wearable sensing gait data, which may lose some useful gait feature information and reduce the classification performance of the model.
The results of selecting the number of neurons in the optimal LSTM model are shown in Figure 3, and it can be seen from Figure 3 that when the number of neurons increases from 16 to 256, the selection of different neurons affects the classification performance of the gait pattern of the model.When the number of neurons is 32, the classifica-tion accuracy is the largest, reaching 92.3 %.When the number of neurons increases from 32 to 256, the classification accuracy decreases significantly.The results show that when the number of neurons is 32. The proposed model can obtain more time-related feature information closely related to gait changes from the local features of wearable sensor gait data space, which significantly improves the classification performance of the proposed model.  (2) Gait classification performance evaluation results of CNN-LSTM model The classification performance evaluation results of the CNN-LSTM gait deep integration learning model based on the optimal parameters taken in this paper are shown in Table 2. It can be seen from Table 2 that the proposed model can identify 6 different gait modes with good classification performance, with an average accuracy of 91.45% and an average recall rate of 91.53%. In comparison, the "lying" gait mode has the highest accuracy rate of up to 99%, which shows that the deep-depth learning model proposed in this paper can effectively obtain the time-space gait characteristic information closely related to the "lying" gait mode from the wearable sensing acceleration and gyroscope gait data, can effectively improve its mode identification performance. However, the accuracy of the "standing" gait mode is the lowest, only 80.94%, and the recall rate of the "sitting" gait mode is the lowest, only 81.06%, and these results show that it is difficult for the model proposed in this paper to obtain time-space gait related characteristic information closely related to the "sitting" and "standing" gait mode from the wearable sensing acceleration and gyroscope gait data, which may be due to the gait data acquisition process of wearable single sensor gait collector is difficult to capture the relevant information of the "sitting, standing" gait mode. In addition, based on the same gait data, this paper selected a gait classification model based on traditional machine learning algorithms (such as decision tree, KNN, support vector machine, etc.) to further evaluates the superior performance of the proposed model, and its comparative classification performance is shown in Table 3. From Table 3, the accuracy, precision, and recall rate of the gait deep integration learning model proposed in this paper were the highest, which can reach 91.5%; Secondly, the accuracy, precision, and recall rate of the KNN gait classification model were about 90%, while the accuracy, precise, and recall rate of the support vector machine were all less than 90%, and the accuracy, precise, and recall rate of the decision tree gait classification model were the lowest, which was only 86%. The above results show that the classification performance of the CNN-LSTM gait deep fusion learning model proposed in this paper is significantly better than that of the traditional machine learning gait classification model, and the fundamental reason is that the model proposed in this paper can make full use of the excellent characteris-tics of the most presentative data obtained by the CNN and LSTM deep-integration learning algorithms, effectively obtain the most representative time-space correlation gait characteristics from wearable sensing acceleration and gyroscope gait data, and significantly improve the gait classification performance. However, the traditional machine learning gait classification model can only obtain local spatial and temporal gait characteristics based on the linear model, and it is difficult to obtain the most representative time-space correlation gait characteristics from wearable sensing acceleration and gyroscope timing gait data, which affects its classification performance.
In addition, in order to further evaluate the effectiveness of the proposed model, based on the above-mentioned same gait data, the proposed model is compared with other traditional deep learning models (such as CNN, RNN [20] , LSTM, GRU [21] and other models), and the comparison results are shown in Table 4. From Table 4, it can be seen that the gait classification performance of the CNN-LSTM model proposed in this paper is significantly better than other traditional deep learning gait classifications performance. In comparison, the gait classification performance of the RNN network model is poor, and the accuracy, precision, and recall rate are only about 70%, which is due to the fact that the RNN network learning model is difficult to obtain the most spatial and time-related gait characteristic information in the gait time series; The accuracy, precision, and recall rate of CNN, GRU and LSTM learning models are about 88%, and although their gait classification performance is better than the RNN network learning model gait classification performance, it is significantly lower than the gait classification performance of the CNN-LSTM fusion learning model mentioned in this paper, and the fundamental reason is that: the gait classification model based on traditional CNN deep learning can only obtain the most representative local spatial gait characteristic information inherent in gait time series data; The gait classification model based on traditional LSTM and GRU deep learning can only obtain the most representa-tive time-correlated gait characteristic information inherent in wearable gait time series data. The limitation of the above two traditional deep learning gait classification models is that difficult to obtain the most representative time-space correlation gait feature information in the wearable gait time series data. However, the CNN-LSTM fusion deep learning gait classification model proposed in this paper can fully integrate CNN and LSTM with excellent characteristics to obtain the inherent spatial and temporal correlation characteristic information of gait time series, effectively obtain the most representative time-space correlation gait feature information of wearable gait time series data, and make up for the limitations of traditional CNN and LSTM deep learning models in obtaining the most representative gait feature information of gait time series data which can effectively improve gait classification performance based on wearable gait sensing data.

Conclusions
In this paper, a new model for gait pattern deep learning fusion discriminant based on wearable sensing data is proposed, which can fully integrate the excellent characteristics of the most representative spatiotemporal characteristics of the data obtained by the convolutional neural network and the long short-term memory neural network deep learning model, and effectively obtain the most representative spatiotemporal correlation gait characteristics from the wearable sensor acceleration and gyroscope gait data, significantly improve the classification performance of wearable gait mode. It provides a reliable reference for further research of wearable multi-sensor gait mode deep learning classification.