{'fraction_non_alphanumeric': 0.031504196255648806, 'fraction_numerical': 0.011168495803744351, 'mean_word_length': 5.008921644685803, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Wearable-based Classification of Running Styles with Deep Learning

Automatic classification of running styles can enable runners to obtain feedback with the aim of optimizing performance in terms of minimizing energy expenditure, fatigue, and risk of injury. To develop a system capable of classifying running styles using wearables, we collect a dataset from 10 healthy runners performing 8 different pre-defined running styles. Five wearable devices are used to record accelerometer data from different parts of the lower body, namely left and right foot, left and right medial tibia, and lower back. Using the collected dataset, we develop a deep learning solution which consists of a Convolutional Neural Network and Long Short-Term Memory network to first automatically extract effective features, followed by learning temporal relationships. Score-level fusion is used to aggregate the classification results from the different sensors. Experiments show that the proposed model is capable of automatically classifying different running styles in a subject-dependant manner, outperforming several classical machine learning methods (following manual feature extraction) and a convolutional neural network baseline. Moreover, our study finds that subject-independent classification of running styles is considerably more challenging than a subject-dependant scheme, indicating a high level of personalization in such running styles. Finally, we demonstrate that by fine-tuning the model with as few as 5% subject-specific samples, considerable performance boost is obtained.


I. INTRODUCTION
Running is a common aerobic exercise which many people worldwide partake in as a sport. It is often considered one of the most historic types of competitive sport, dating back centuries [1]. Despite its pervasiveness, running can be performed in a wide variety of different 'styles' [2], [3], often as a result of different physiological/biomechanical factors [4], training [5], and even the intention of the runner. These variations can play major roles in energy expenditure [6], speed/endurance [7], and even injury [2], [8], [9].
Based on the above, we believe it is highly valuable and beneficial to develop systems capable of automatically detecting different running styles. Such a system can be used for injury analysis, coaching and training, and other applications. Moreover, we believe the use of wearable sensors for such a system, as opposed to cameras for instance, will allow for more detailed, exact, and targeted monitoring as well as lack of occlusions of key body parts.
The notion of using wearables and machine learning to classify different activities has been widely studied in the literature. For instance, in [10], hand-crafted features were 1 used along with an ensemble of Support Vector Machine (SVM) learners to classify daily activities using wearable data. Similar approaches have been used in [11], [12], and [13], where different classical machine learning solutions have been used. In [14], deep learning, in the form of a Convolutional Neural Network (CNN) was used to perform end-to-end classification of different activities. In [15], the learned representations from a CNN were fed to a Long Short-Term Memory (LSTM) network to classify actions. More recently, self-supervised solutions have been proposed in [16], [17], [18], and [19] to reduce reliance on labels and learn more generalized representations by means of pretext training with a number of transformations applied to wearable data. However, despite the rich available literature on classification of different activity classes, the notion of detecting specific running styles with machine learning, which may be manifested as a result of very minor and subtle changes in running patterns, has not yet been explored.
To tackle the problem of automatic wearable-based running style classification, we design a system for automatic wearable-based classification of different running styles with deep learning. We first record a rich dataset of different subjects running in a number of different pre-defined running styles. The data are recorded using several wearable sensors placed strategically on different parts of the body including each foot, each medial tibia, and on the lower back. We then use this dataset to train deep neural networks for classification of the different styles. Our results show that our proposed CNN can accurately classify the different running styles and outperforms some classical machine learning models, namely support vector machine (SVM), naive Baysed (NB), decision tree (DT), random forest (RF). Interestingly, however, we find that the performance considerably drops when the experiments are performed in a leave-one-subjectout manner, even when different CNN architectures are explored. This finding points to the fact that running styles, even in pre-defined categories, are highly subject-specific and vary across different runners.
Lastly, we demonstrate that by fine-tuning the model with small portions of unseen subjects' samples, a more robust performance is achieved.

II. METHODOLOGY
A. Dataset 10 healthy and unimpaired runners (5 women and 5 men), ranging in age between 20 to 26 years, were recruited for this study [20]. One of the inclusion criteria was for participants to have been running around 10 km per week over the last three months leading to the data collection session. For each subject, a set of 5 IMeasureU [21] Inertial Measurement Units (IMUs) sensors were placed on each foot (top of the shoe, attached through laces), each medial tibia (attached with elastic Velcro straps), and one at the centre of mass on the lower back (attached with Velcro to a torso wrap) to record the accelerations and angular changes of their body segments when running. The accelerometer data are sampled at 500 Hz and collected by a personal computer via Bluetooth. For each subject, their height, weight, age, and sex were also recorded. For all the experiments, an instrumented treadmill was used. First, the participants were asked to start running on the treadmill at 1.5 m/s and then gradually increase their speed. This helped the runners to warm up and also was used to determine the comfortable speed for each runner to perform the experiments. Eight different running styles were considered for this experiment: (1) Egg-Beater Gait (where instead of only moving forward, the leg also travels laterally to the side); (2) Bouncing Gait (where the vertical bounce after each step is higher than normal); (3) Heel Strike (where landing occurs on the heels as opposed to mid-foot landing); (4) Toe Strike (where landing occurs on the fore-foot); (5) Lengthened Strides (where the strides are abnormally longer than normal); (6) Shortened Strides (where the strides are abnormally shorter than normal); (7) Wide Stance (where the horizontal space between the two feet is wider than normal); and (8) Narrow Stance (where the horizontal space between the two feet is shorter than normal). These particular classes were selected as they encapsulate exaggerated common variations in running styles among people. Each subject was asked to perform these running styles for approximately 5 minutes after each style was verbally described to them. This approach allowed for subjects to perform slight variations of each style given their own comfort and physiology. Ethics approval was secured from Queen's General Research Ethics Board.

B. Proposed Model
In this study, we propose a CNN-LSTM architecture to classify different running styles based on input multi-sensor accelerometer data. The first part of our model is a CNN which learns to automatically extract representations from the 3-D accelerometer signals. This network consists of 3 blocks with 2 1-D convolutional layers in each block. Figure 1(a) demonstrates the CNN component of our model. The kernel size is set to 3 and the numbers of filters for the convolutional layers in each block are 64, 128, and 256, respectively. These blocks are then followed by a 1D maxpooling layer with a pool size of 2 and a flatten layer to generate the output features from the input signals.
Following the CNN, a Recurrent Neural Network is used in the form of a 2-layer Bidirectional LSTM (BLSTM), each consisting of 256 hidden units. The BLSTM aims to learn the temporal traits and relationships between the representations extracted by the CNN from input raw accelerometer data in both forward and backward directions. The output of the final bidirectional layer is then fed to 2 fully connected layers (where the first one is followed by ReLU activation) of sizes 200 and 8 (number of running classes) and a softmax layer. A similar CNN-LSTM architecture is used for the data from each sensor, and the final outputs of the 5 CNN-LSTM networks are fused through score-level fusion (averaging the output probabilities). Figure 1(b) presents the overall architecture of our proposed solution. The dashed green rectangle in the figure illustrates the details of the network used for each sensor.
To feed each CNN-LSTM model, we divide the running data into segments of 10 seconds in duration, with 50% overlap. As shown in Figure 1, the input of the network is a 10-second 3-D acceleration signal collected from each of the five IMUs placed on different parts of the body. Each 10-second input signal is segmented into 8 successive sub-segments of 1.25 seconds in duration with no overlap. The 1.25 second sub-segments are fed to the CNN network to generate vector representations, to subsequently feed 8 LSTM cells organized in 2 bidirectional layers.

C. Implementation and Evaluation
We implement our proposed model using Keras with TensorFlow backend on an NVIDIA GeForce RTX 2080 Ti GPU. The models are trained over 300 epochs with a batch size 64 and Adam optimizer with a learning rate of 0.0002.
In order to evaluate our method, we use 2 schemes: kfolds cross-validation and Leave-One-Subject-Out (LOSO). For k-folds, we randomly picked 20% of the segments for testing, while using the rest for training. We carried out this approach for 5 trials and presented the means and standard deviations of accuracy and F1 scores among all 5 folds. In each trial, 10% of the training set was allocated for validation. In LOSO, for each trial, 20% of the subjects were selected for testing and the remaining subjects were set to train the model. Similar to the previous approach, 10% of the training data was set aside for validation.

D. Baselines
In order to better evaluate the robustness of our proposed method, several common classification methods are applied for comparison. Specifically, 4 classical machine learning classifiers are used. These classifiers are: Naive Bayes, Decision Tree, SVM, and Bagged Tree Ensemble (BTE). We extract 8 common manual features (mean, standard deviation, minimum, maximum, root mean square, skewness, kurtosis, and peak-to-peak time) from each segment to train these classical models.
The hyperparameters for each model are tuned with the help of Bayesian optimization. For SVM, cubic kernel and one-vs-one scheme were the best fits for all the sensors, with the exception of the LFoot sensor where a Gaussian kernel function and one-vs-all scheme showed the best results. For the Decision Tree, maximum deviance reduction was set as the Split criterion for all sensors except for the COM sensor which used the Towing rule.
In addition to the 4 classical machine learning methods, a CNN was designed to automatically extract the features from each input signals and classify the running styles

III. RESULTS
In this section, we present the results of our experiments described in the previous section. Our proposed CNN-LSTM method outperforms all the baselines in both k-fold and LOSO schemes. Table I presents the accuracy and F1 scores for all the models with k-fold cross-validation. It can be observed that the proposed CNN-LSTM model obtains the best results achieving an accuracy and F1-score of 0.931±0.008 and 0.932±0.007 respectively. Among the baseline models, the CNN shows the best performance, outperforming the classical machine learning solutions. Exploring the performance of the model on different parts of the body (sensor locations) indicates that all the sensors provide effective information for classification of running styles. Moreover, we observe that by fusing the 5 sensors pipelines, a boost in performance is achieved, demonstrating that different parts of the body contain complimentary information that can be exploited for automatic classification of running styles. We also present the confusion matrix for this model in Figure 2, where we observe that different running styles have been classified consistently across the board. Meanwhile, the 'wide-stance' style proves the easiest to classify whereas the 'bouncing' style proves the most difficult.
The results of the models in the LOSO scheme are presented in Table II proposed CNN-LSTM model outperforms the baselines, the overall performance of all the models, both baselines and the proposed, significantly drop. It can therefore be inferred that variations in running styles are highly subject-specific, making it difficult for automatic subject-independent classification, highlighting the need for solutions capable of learning generalized representations via transfer learning or self-supervised learning. Moreover, the fact that subjects performed the running styles based on their comfort could be another contributing factor to this matter. To further evaluate this matter, we fine-tune both CNN and CNN-LSTM models with a small fraction of test data (5%). As observed in Table II, for both models, the performance is significantly improved, reaching an accuracy of 0.618 for CNN model and 0.714 for CNN-LSTM model. The results of fine-tuning the proposed CNN-LSTM model with various portions of test data are presented in Table III. As expected, increasing the amount of data used for fine-tuning improves the performance of the model, while we observe that utilizing even a very small amount of user-specific data (2%), results in a significant boost in performance.

IV. CONCLUSIONS AND FUTURE WORK
In this study, we implemented a system for classifying running style variations. For this purpose, we collected data using 5 wearable accelerometers from 10 healthy runners while performing 8 different running styles on a treadmill. We then proposed a deep CNN-LSTM architecture to detect these target classes by learning to automatically extract effective representations followed by learning temporal dependencies in the time-series. We compared the performance of our solution to a number of baseline techniques, where CNN-LSTM model showed the best results. Our findings suggest that even pre-defined running styles are personalized for different people and based on comfort and physiology, making it difficult for machine learning and deep learning methods to automatically classify them, however fine-tuning the model with a small portion of each subject's samples, significantly improved the LOSO performance.
For future work, we will explore approaches such as transfer learning, self-supervised learning, and contrastive learning, that have proven effective in enabling deep learning solutions to extract highly generalized features from input data. Moreover, domain transfer techniques may be explored to perform cross-subject domain adaptation.