{'fraction_non_alphanumeric': 0.03342165695106872, 'fraction_numerical': 0.041618088676912204, 'mean_word_length': 4.87749169435216, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DeepSleep: Fast and Accurate Delineation of Sleep Arousals at Millisecond Resolution by Deep Learning

Sleep arousals are transient periods of wakefulness punctuated into sleep. Excessive sleep arousals are associated with many negative effects including daytime sleepiness and sleep disorders. High-quality annotation of polysomnographic recordings is crucial for the diagnosis of sleep arousal disorders. Currently, sleep arousals are mainly annotated by human experts through looking at millions of data points manually, which requires considerable time and effort. Here we present a deep learning approach, DeepSleep, which ranked first in the 2018 PhysioNet Challenge for automatically segmenting sleep arousal regions based on polysomnographic recordings. DeepSleep features accurate (area under receiver operating characteristic curve of 0.93), high-resolution (5-millisecond resolution), and fast (10 seconds per sleep record) delineation of sleep arousals.


Abstract 27
Sleep arousals are transient periods of wakefulness punctuated into sleep. Excessive sleep arousals are 28 associated with many negative effects including daytime sleepiness and sleep disorders. High-quality 29 annotation of polysomnographic recordings is crucial for the diagnosis of sleep arousal disorders. Currently, 30 sleep arousals are mainly annotated by human experts through looking at millions of data points manually, Sleep is important for our overall health and quality of life 1 . Inadequate sleep is often associated with many 39 negative outcomes, including obesity 2 , irritability 2,3 , cardiovascular dysfunction 4 , hypotension 5 , impaired 40 memory 6 and depression 7 . About one third of the general population in the United States are affected by 41 insufficient sleep 8 . The prevalence of inadequate sleep results in large economic costs 9 and continues to 42 increase in various nations 10,11 . Spontaneous sleep arousals, defined as brief intrusions of wakefulness into 43 sleep 12 , are a common characteristic of brain activity during sleep. Excessive arousals due to disturbances 44 can be harmful, resulting in fragmented sleep, daytime sleepiness and sleep disorders 13,14 . There are 45 different types of arousing 15 stimulus, including obstructive sleep apneas or hypopneas, respiratory effort-46 related arousals (RERA), hyperventilations, bruxisms (teeth grinding), snoring, vocalizations, and leg 47 movements. Together with sleep stages (wakefulness, stage1, stage2, stage3, and rapid eye movement), 48 sleep arousals are labeled through visual inspections of polysomnographic recordings according to the 49 American Academy of Sleep Medicine (AASM) scoring manual 16 . Of note, an 8-hour sleep record sampled 50 at 200Hz with 13 different physiological measurements contains a total of 75 million data points. It takes 51 hours to manually annotate such a large-scale sleep record. 52 Many research efforts have been made in developing computational methods for automatic arousal 53 detection based on polysomnographic recordings 17-21 . These methods mainly focus on 30-second epochs, 54 and extract statistical features in the time and frequency domains through Fourier transform or in-house 55 feature engineering. These features and/or raw signals are subsequently fed into machine learning models 56 to predict sleep arousals. However, due to the large differences of datasets and evaluation metrics used in 57 previous studies, it remains unknown how to build an accurate and robust model to quickly delineate all 58 sleep arousal events within a sleep record at a high resolution. In particular, how to preprocess the raw data 59 or extract features before training models? Which types of machine learning models are well suited? What 60 is the optimal input length (e.g. 30-second epochs or full-length records)? Which types of physiological 61 signals should be used? 62 Here we investigate these questions and describe a novel deep learning approach, DeepSleep, for automatic 63 detection of sleep arousals. This approach ranked first in the 2018 "You Snooze, You Win" 64 PhysioNet/Computing in Cardiology Challenge 22 , in which state-of-the-art computational methods were 65 systematically evaluated for predicting non-apnea sleep arousals on a large held-out test dataset 23 . The 66 workflow of DeepSleep is schematically illustrated in Fig. 1. We built a deep convolutional neural network 67 (CNN) to capture long-range and short-range interdependencies between time points across an entire sleep 68 record. Information at different resolutions and scales was integrated to improve the performance. 69 Intriguingly, we found that similar EEG and EMG channels were interchangeable, which was used as a 70 special augmentation in our approach. DeepSleep is able to delineate the sleep arousal profile of a sleep 71 record at 5-millisecond resolution within 10 seconds. Location. The 13-channel polysomnogram monitored multiple body functions, including brain activity 78 (EEG), eye movement (EOG), muscle activity (EMG), and heartbeat (ECG airflow; one measure of oxygen saturation (SaO 2 ); one electrocardiogram (ECG). Each time point in the 92 polysomnographic record was labeled as "Arousal" or "Sleep" by sleep experts, excluding some non-93 scoring regions such as apnea or hypopnea arousals. To exploit the information of the training records, we 94 employed a nested train-validate-test framework, in which 60% of the data was used to train the neural 95 network, 15% of the data was used to validate for parameter selection and 25% of the data was used to 96 evaluate the performance of the model (Cross-validation in Fig. 1). To capture the long-range and short-97 range information at different scales, we adapted a classic neural network (Model in Fig. 1), U-Net, which 98 was originally designed for image segmentation 24 . Multiple data augmentation strategies, including 99 swapping similar polysomnographic channels, were used to expand the training data space and enable the 100 generalizability of the model. Finally, the prediction performance was evaluated by the area under receiver 101 operating characteristic curve (AUROC) and the area under precision-recall curve (AUPRC) on the held-102 out test dataset of 989 records (Evaluation in Fig. 1) during the challenge. The apneic and non-apneic arousal events overwrite sleep stages (N1, N2, N3, REM). (B) The relationship 108 between the number of sleep arousals (y-axis) and the percentage of total sleep arousal time over total sleep 109 time (x-axis) in the 994 sleep records. In general, more arousal events lead to longer accumulated arousal 110 time and the correlation is significantly strong. (C) The length of sleep (x-axis) has no significant correlation 111 with the accumulated length of sleep arousals (y-axis). 112

Highly heterogeneous sleep records among individuals 113
By investigating the annotations of these sleep records, we found high levels of heterogeneity among 114 individuals. In Fig. 2A, we randomly selected sleep records of 20 individuals and presented the annotations 115 in different colors. There are 8 major annotation categories: "Arousal", "Undefined", "REM" (Rapid Eye 116 Movement), "N1" (Non-REM stage 1), "N2" (Non-REM stage 2), "N3" (Non-REM stage 3), "Wake" and 117 "Apnea". The distribution of these categories differs dramatically among individuals (different colors in 118 Fig. 2A). Clearly, different individuals display distinct patterns of sleep, including the length of total sleep 119 time and multiple sleep stages. Notably, the sleep arousal regions are relatively short and sparsely 120 distributed along the entire record for most individuals (yellow regions in Fig. 2A). 121 We further investigated the occurrence of arousals and found that the median number of arousals during 122 sleep was 29, indicating the prevalence of sleep arousals. A total of 43 individuals (4.33%) had solid sleep 123 without any arousal, whereas 82 individuals (8.25%) had more than 100 arousals during their sleep (y-axis 124 in Fig. 2B), lasting around 10% of the total sleep duration (x-axis in Fig. 2B). In addition, there was no 125 significant correlation between the total sleep time and the total length of sleep arousals (Fig. 2C), which 126 was expected since the quality of sleep is not determined by sleep length. In summary, the intrinsically high 127 heterogeneity of sleep records across individuals rendered the segmentation of sleep arousals a very difficult 128 problem. consistently achieved better performances (Fig. S1). Therefore, we used the entire sleep record as input to 144 make predictions, instead of small segments of a sleep record. 145 To learn the long-range association between data points across different time scales (second, minute, and 146 hours), we develop an extremely deep convolutional neural network, which contains a total of 35 147 convolutional layers (Fig. 3A). This network architecture has two major components, the encoder and the 148 decoder. The encoder takes a full-length sleep record of 2 23 = 8,388,608 time points and gradually encrypts 149 the information into a latent space (the red trapezoid in Fig. 3A). Sleep recordings were centered, regardless 150 of their original lengths, within the 8-million input space by filling in with zeros on their extremes. To be 151 specific, the convolution-convolution-pooling (hereafter referred to as "ccp") block is used to gradually 152 reduce the size from 2 23 = 8,388,608 to 2 8 = 256 (Fig. 3B top). Meanwhile, the number of channels gradually 153 increases from 13 to 480 to encode more information, compensating the loss of resolution in the time 154 domain. In each convolutional layer, the convolution operation is applied on the data along the time axis to 155 aggregate the neighborhood information. Since the sizes of data in these convolutional layers are different, 156 the encoded information is unique within each layer. For example, in the input layer, 10 successive time 157 points sampled at 200Hz correspond to a short time interval of 10/200=0.05 seconds, whereas in the center 158 layer (size = 2 8 ), 10 time points correspond to a much longer time interval of 0.05 * 2 23-8 = 1,638 seconds, 159 nearly 30 minutes. Therefore, this deep encoder architecture allows us to capture and learn about the 160 interactions across data points at multiple time scales. The relationship between length of segments and the 161 corresponding time can be found in Table S1. 162 Similar to the encoder, the second component of our network architecture is a decoder to decrypt the 163 compressed information from the center latent space. In contrast to the "ccp" block, the convolution-164 convolution-upscaling (hereafter referred to as "ccu") block is used (Fig. 3B bottom), which gradually 165 increases the size and decreases the number of channels of the data (the purple trapezoid in Fig. 3A). In 166 addition, concatenation is used to integrate the information from both the encoder and the decoder at each 167 time scale (green horizontal arrows in is observed between AUROCs and AUPRCs with a significant p-value < 0.001. The 95% percent 176 confidence interval is shown as the yellow bend. The baselines of random predictions are shown as red 177 dashed lines. The prediction (C) AUPRCs and (D) AUROCs of models using different resolution or 178 strategies were calculated. The "1/8_no_swap" model corresponds to the model using the "1/8" resolution 179 records as input without any channel swapping, whereas the "1/8", "1/2" and "full" models use the strategy 180 of swapping similar polysomnographic channels. The final "1/8+1/2+full" model of DeepSleep is the 181 ensemble of models at 3 different resolutions, achieving the highest AUPRC of 0.550 and AUROC of 0.927. 182 183

Deep learning enables accurate predictions of sleep arousals 184
By capturing the information at multiple resolutions, DeepSleep achieves high performance in automatic 185 segmentation of sleep arousals. Since deep neural networks are iteration-based machine learning 186 approaches, a validation subset is used for monitoring the underfitting or overfitting status of a model and 187 approximating the generalization ability on unseen datasets. A subset of 15% randomly selected records 188 was used as the validation set during the training process (Cross-validation in Fig. 1) and the cross entropy 189 was used to measure the training and validation losses (see details in Materials and Methods). The 13 190 polysomnographic channels complemented each other and using all of them instead of one type of these 191 signals enabled the neural network to capture interactions between channels and achieved the highest 192 performance ( Fig. S2A-B). We developed three basic models called "1/8", "1/2" and "full", according to 193 the resolution of the neural network input. The "full" resolution means that the original 8-million (2 23 = 194 8,388,608) length data were used as input. The "1/2" or "1/8" resolution means that the original input data 195 were first shrunk to the length of 4-million (2 22 ) or 1-million (2 20 ) by averaging every 2 or 8 successive time 196 points, respectively. We observed similar validation losses of the "full", "1/2" and "1/8" models (solid lines 197 in Fig. 4A). The final evaluation was based on the AUROC and AUPRC scores of predicting 25% of the 198 data. In Fig. 4B, each blue dot represented one sleep record and we observed a significant yet weak 199 correlation = 0.308 between the AUROCs and AUPRCs. The baselines of random predictions were shown 200 as red dashed lines. Notably, the AUPRC baseline of 0.072 corresponded to the ratio of the average total 201 sleep arousal length over the total sleep time, which was considerably low and made it a hard task due to 202 the intrinsic sparsity of sleep arousal events. 203 To build a robust and generalizable model, multiple data augmentation strategies were used in DeepSleep.

204
After carefully examining the data, we found that signals belonging to the same physiological categories 205 were very similar and synchronized, including two EMG channels and six EEG channels (see Data in Fig.  206 1). We applied a novel augmentation strategy by randomly swapping these similar channels during the 207 model training process, assuming that these signals were interchangeable in determining sleeping arousals. 208 There are three EMG channels but EMG-chin were not considered in this swapping strategy due to its 209 differences from the other two EMG (ABD and chest) channels (see Data in Fig. 1). This channel swapping 210 strategy was bold but effective, adapting which largely improved the prediction performance 211 ("1/8_no_swap" versus "1/8" in Fig. 4C-D). In addition, we multiplied the polysomnographic signals by a 212 random number between 0.90 and 1.15 to simulate the inherent fluctuation and noise of the data. Other 213 augmentations on the magnitude and time scale were also explored ( Fig. S2C-D). Furthermore, to address 214 the heterogeneity and batch effects among individuals, we quantile normalized each sleep record to a 215 reference, which was generated by averaging all the records. This step effectively removed the biases 216 introduced by the differences of individuals and instruments, and Gaussian normalization was also tested 217 and had slightly lower performance (Fig. S2E-F). Finally, we assembled the predictions from the "1/8", 218 "1/2" and "full" resolution models as the final prediction in DeepSleep (red violin plots in Fig. 4C-D). 219 We further compared different machine learning models and strategies in segmenting sleep arousals. We 220 first tested a classical model, logistic regression, and found that our deep learning approach had a much 221 higher performance (Fig. S2G-H). It has also been reported that neural network approaches significantly 222 outperformed classical machine learning methods, including random forest, logistic regression 25 , support 223 vector machine, and linear models 26  (GRU) into DeepSleep did not improve the performance (Fig. S3B-D). In terms of input length, increasing 227 input length significantly improved the performance, and full-length records were used by three teams (blue 228 blocks in Fig. S3A). We also compared DeepSleep with recent state-of-the-art methods in sleep stage 229 scoring. These methods extracted features from 30-second epochs through short-time Fourier transform 230 (STFT) 27,28 or Thomson's multitaper 25,29 . They were originally designed for automatic sleep staging and 231 we applied them to the task of detecting sleep arousals on the same 2018 PhysioNet data. Although these 232 methods performed well in sleep stage scoring, they were not well suited for arousal detection (Fig. S3E-233 F). Deep learning approaches can model informative features in an implicit way without tedious feature 234 crafting 30 , and neural networks using raw data as input were frequently used by half of the top 10 teams 235 (orange blocks in Fig. S3A). 236 To comprehensively investigate the effects of various network structures and parameters on predictions, we 237 further performed experiments with different modifications, including shallow neural network (Fig. S4A-238 B), average pooling (Fig. S4C-D), large convolution kernel size (Fig. S4E-F), and loss functions (Fig.  239  S4G-H). These modifications had either similar or lower prediction performances. We concluded that the 240 neural network architecture and augmentation strategies in DeepSleep were optimized for the current task 241 of segmenting sleep arousals. Subsequent analysis of the relationships between the prediction performance 242 and the number of arousal were investigated (Fig. S5A-B). As we expected, the prediction AUPRC was 243 correlated with the number of arousals in a sleep record. The individuals who had more sleep arousals 244 during sleep were relatively easier to predict. Moreover, we tested the runtime of DeepSleep with Graphics 245 Processing Unit (GPU) acceleration and segmenting sleep arousals of a full sleep record can be finished 246 within 10 seconds on average (Fig. S5C-D). The time cost of DeepSleep is much lower than that of manual 247 annotations, which requires hours for one sleep record. 248 In addition to the 2018 PhysioNet dataset, we further validated our method on the large publicly available 249 Sleep Heart Health Study (SHHS) dataset, which contains 6,441 individuals in SHHS visit 1 (SHHS1) and 250 3,295 individuals in SHHS visit 2 (SHHS2) 31 . The SHHS is a multi-center cohort study, including 251 participants from multiple different cohorts and the polysomnograms were annotated by sleep experts from 252 different labs (https://sleepdata.org/datasets/shhs). The recording montages and signal sampling rates of 253 SHHS1 and SHHS2 were quite different. For both SHHS1 and SHHS2, we randomly selected 1,000 254 recordings, which was comparable to the number of recordings (n=994) in the PhysioNet training dataset. 255 Then we applied DeepSleep pipeline to train, validate, and test models on SHHS1 and SHHS2 datasets 256 individually. We observed similar performances of detecting sleep arousals on the PhysioNet, SHHS1, and 257 SHHS2 datasets in Fig. S6A-B, demonstrating the robustness of our DeepSleep method.

259
In the clinical setting, both apneic and non-apneic arousal are very important. We have therefore built neural 260 network models for detecting apnea, in addition to the model for detecting non-apneic arousals, which was 261 originally designed during the 2018 PhysioNet challenge. Specifically, we applied DeepSleep pipeline to 262 the PhysioNet data and built three types of models for (1) detecting apneic arousals; (2) detecting non-263 apneic arousals; and (3) detecting all arousals (apneic and non-apneic arousals). DeepSleep is able to detect 264 both apneic and non-apneic arousals (Fig. S6C-D). 265 266 annotations by introducing the probability system. 295

296
Discussion 297 In this study, we created a deep learning approach, DeepSleep, to automatically segment sleep arousal 298 regions in a sleep record based on the corresponding polysomnographic signals. A deep convolutional 299 neural network architecture was designed to capture the long-range and short-range interactions between 300 data points at different time scales and resolutions. Unlike classical machine learning models 32 , deep 301 learning approaches do not depend on manually crafted features and can automatically extract information 302 from large datasets in an implicit way 33 . Using classical approaches to define rules and craft features for 303 modelling sleep problems in real life would become much too tedious. In contrast, without assumptions and 304 restrictions, deep neural networks can approximate complex mathematical functions and models to address 305 those problems. Currently, these powerful tools have also been successfully applied to biomedical image 306 analysis and signal processing 34,35 . Compared with classical machine learning models, deep learning is a 307 "black box" method which is relatively hard to interpret and understand. Meanwhile, deep learning 308 approaches usually requires more computational resources such as GPUs, whereas most classical machine 309 learning models can run on common CPUs. 310 Overfitting is a common issue in deep learning models, especially when the training dataset is small and 311 the model is complex. Even if we use a large dataset and perform cross-validation, we will gradually and 312 eventually overfit to the data. This is because each time we evaluate a model using the internal test set, we 313 probe the dataset and fit our model to it. In contrast to previous studies, the 2018  can be adapted to other segmentation tasks such as sleep staging. A multi-tasking learning approach can be 320 further implemented as the outputs of U-Net to directly segment multiple sleep stages simultaneously based 321 on polysomnograms. 322 An interesting observation is that when we used records of different lengths as input to train deep learning 323 models, the model using full-length records largely outperformed models using short periods of records.

324
This observation brings about the question of how to accurately detect sleep arousals based on 325 polysomnography. Current standards mainly focus on short time intervals of less than one minute 16 , yet 326 the segmentations among different sleep experts are not very consistent in determining sleep arousals. One 327 reason is that it is hard for humans to directly read and process millions of data points at once. In contrast, 328 computers are good at processing large-scale data and discover the intricate interactions and structures 329 between data points across seconds, minutes and even hours. Our results indicate that sleep arousal events 330 are not be solely determined by the local physiological signals but associated with much longer time 331 intervals even spanning hours. It would be interesting to foresee the integration of computer-assisted 332 annotations to improve definitions of sleep arousals or other sleep stages.

Partition of the training, validation and testing sleep records 359
The 994 sleep records were randomly partitioned into three sets: 60% of them as the training set, 15% of 360 them as the validation set and 25% of them as the testing set. The validation set was used for monitoring 361 the training-validation losses and avoiding the problems of overfitting or underfitting. 362

Gaussian normalization 363
The Gaussian normalization is calculated by which is also in the increasing order. For example, x k ik will be replaced by ref k . Then we changed the order 381 back and mapped ref k to its original position ik. After this quantile normalization, the overall distribution 382 of the input vector has been mapped to the distribution of the reference vector. The reference vector was 383 pre-calculated by averaging all the sorted recordings in the training dataset. We quantile normalized each 384 recording to the same reference to address potential batch and cohort effects. Each polysomnographic 385 channel was normalized individually. 386 387 AUROC and AUPRC 388 Since sleep arousal events are extremely rare (<10% in terms of length), the performances of different 389 methods are not apparent in the Receiver Operating Characteristic (ROC) curve, where the y-axis is the 390 True Positive Rate (TPR) and the x-axis is the False Positive Rate (FPR). The TPR and FPR are defined as 391 where TP is True Positive, FN is False Negative, FP is False Positive, and TN is True Negative. This is 394 because when the number of negative events ("Sleep"; 92.8%), or TN, is much larger than the positive ones 395 ("Arousal"; 7.2%), the FPR is always very small and will barely change even if a poor model makes many 396 FP predictions. Therefore, in addition to the commonly used AUROC, we evaluated our model and various 397 strategies using ARPRC 36,37 . In the Precision-Recall space, the Precision and Recall are defined as 398 The Precision is very sensitive to FP when the number of TP is relatively small. Therefore, the AUPRC 401 metric is able to distinguish the performances in highly unbalanced data such as the annotations of sleep 402 arousals. 403

Convolutional neural network architectures 404
The classic U-Net architecture was adapted in DeepSleep. The original U-Net is a 2D convolutional neural 405 network designed for 2D image segmentation 24 . We transformed the structure into 1D for the time- In addition, batch normalization was used after each convolutional layer. In the final output layer, we used 415 the sigmoid activation unit defined as 416 where x is the input to a neuron and f(x) is the output. During the training process, the Adam optimizer was 418 used with the learning rate of 1e-4 and the decay rate of 1e-5. 419 Other network structures were also tested, including Long Short-Term Memory (LSTM) and Gated 420 Recurrent Unit (GRU). They have similar performances. Therefore, we kept the U-Net based structure. 421

Training Losses 422
The cross entropy loss, or log loss, was used for model training in DeepSleep. The cross entropy loss is 423 defined as 424 where " is the gold standard label of sleep=0 or arousal=1 at time point i, I is the prediction value at time 426 point i, N is the total number of time points, is the vector of the gold standard labels and is the vector 427 of predictions. Ideally, an "AUPRC loss" should be used for optimizing the prediction AUPRC. However, 428 the "AUPRC loss" doesn't exist because the AUPRC function is not mathematically differentiable, which 429 is required in the neural network model training through the back-propagation algorithm 38 . Therefore, we 430 need to use cross-entropy loss to approximate the "AUPRC loss". Another option is using the Sorensen-431 dice coefficient defined as 432 of predictions. We have tested the cross-entropy loss, the Sorensen dice loss and combining these two losses. 436 Using the cross-entropy loss achieved the best performance in DeepSleep. 437

Overall AUPRC and AUROC 438
The overall AUPRC, or the gross AUPRC, is defined as   input, all 13 channels were used together as input features in (C) and (D). Longer input lengths achieved 659 higher AUPRCs and AUROCs. The value above each violin is the overall AUPRC/AUROC, which is 660 different from the simple mean or median value. The overall AUPRC/AUROC considers the length of each 661 record and longer records contribute more to the overall AUPRC/AUROC (see details in Methods -662 Overall AUPRC and AUROC). 663 Study visit 1 (SHHS1), and SHHS2 datasets were compared. The performance on these three datasets was 746 comparable. We further tested the prediction (C) AUPRCs and (D) AUROCs of DeepSleep on apneic, non-747 apneic, and all (both apneic and non-apneic) arousals. The value above each violin is the overall 748 AUPRC/AUROC, which is different from the simple mean or median value. The overall AUPRC/AUROC 749 considers the length of each record and longer records contribute more to the overall AUPRC/AUROC (see 750 details in Methods -Overall AUPRC and AUROC). 751