{'fraction_non_alphanumeric': 0.03889572229055624, 'fraction_numerical': 0.014486811534782014, 'mean_word_length': 4.838812639642515, 'pattern_counts': {'":': 1, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Contrastive Learning for Unsupervised Domain Adaptation of Time Series

Unsupervised domain adaptation (UDA) aims at learning a machine learning model using a labeled source domain that performs well on a similar yet different, unlabeled target domain. UDA is important in many applications such as medicine, where it is used to adapt risk scores across different patient cohorts. In this paper, we develop a novel framework for UDA of time series data, called CLUDA. Specifically, we propose a contrastive learning framework to learn contextual representations in multivariate time series, so that these preserve label information for the prediction task. In our framework, we further capture the variation in the contextual representations between source and target domain via a custom nearest-neighbor contrastive learning. To the best of our knowledge, ours is the first framework to learn domain-invariant, contextual representation for UDA of time series data. We evaluate our framework using a wide range of time series datasets to demonstrate its effectiveness and show that it achieves state-of-the-art performance for time series UDA.


Introduction
Many real-world applications of machine learning are characterized by differences between the domains at training and deployment [22,27]. Therefore, effective methods are needed that learn domain-invariant representations across domains. For example, it is well known that medical settings suffer from substantial domain shifts due to differences in patient cohorts, medical routines, reporting practices, etc. [15,57]. Hence, a machine learning model trained for one patient cohort may not generalize to other patient cohorts. This highlights the need for effective domain adaptation of time series.
In contrast, comparatively few works have focused on UDA of time series. Here, previous works utilize a tailored feature extractor to capture temporal dynamics of multivariate time series, typically through recurrent neural networks (RNNs) [36], long short-term memory (LSTM) networks [5], and convolutional neural networks [29, 51,52]. Some of these works minimize the domain discrepancy of learned features via adversarial-based methods [36, 51,52] or restrictions through metric-based methods [5,29].
Another research stream has developed time series methods for transfer learning from the source domain to the target domain [13,14,26,48,[54][55][56]. These methods pre-train a neural network model via contrastive learning to capture the contextual representation of time series from unlabeled source domain. However, these methods operate on a labeled target domain, which is different from UDA.
To the best of our knowledge, there is no method for UDA of time series that captures and aligns the contextual representation across source and target domains.
In this paper, we propose a novel framework for unsupervised domain adaptation of time series data based on contrastive learning, called CLUDA. Different from existing works, our CLUDA framework aims at capturing the contextual representation in multivariate time series as a form of high-level features. To accomplish this, we incorporate the following components: (1) We minimize the domain discrepancy between source and target domains through adversarial training. (2) We capture the contextual representation by generating positive pairs via a set of semantic-preserving augmentations and then learning their embeddings. For this, we make use of contrastive learning (CL). (3) We further align the contextual representation across source and target domains via a custom nearest-neighborhood contrastive learning.
We evaluate our method using a wide range of time series datasets. (1) We conduct extensive experiments using established benchmark datasets WISDM [28], HAR [1], and HHAR [42]. Thereby, we show our CLUDA leads to increasing accuracy on target domains by an important margin. (2) We further conduct experiments on two large-scale, real-world medical datasets, namely MIMIC-IV [24] and AmsterdamUMCdb [45]. We demonstrate the effectiveness of our framework for our medical setting and confirm its superior performance over state-of-the-art baselines. In fact, medical settings are known to suffer from substantial domain shifts (e. g., due to different patient cohorts, medical routines, reporting practices, etc., across health institutions) [15,32,57]. This highlights the relevance and practical need for adapting machine learning across domains from training and deployment.
Contributions: 1 1. We propose a novel, contrastive learning framework (CLUDA) for unsupervised domain adaptation of time series. To the best of our knowledge, ours is the first that learns a contextual representation of multivariate time series to preserve information on labels.

Problem Definition
We consider a classification task for which we aim to perform unsupervised domain adaptation of time series. Specifically, we have two distributions over the time series from the source domain D S and the target domain D t . In our setup, we have labeled i.i.d. samples from the source domain given by S = {(x s i , y s i )} Ns i=1 ∼ D S , where x s i is a sample of the source domain, y s i is the label for the given sample, and N s is the overall number of i.i.d. samples from the source domain. In contrast, we have unlabeled i.i.d. samples from the target domain given by T = {x t i } Nt i=1 ∼ D T , where x t i is a sample of the target domain and N t is the overall number of i.i.d. samples from the target domain. For brevity, we omit the index i (i. e., x s instead of x s i ) when it is clear from the context. In this paper, we allow for multivariate time series. Hence, each x i (either from the source or target domain) is a sample of multivariate time series denoted by where T is the number of time steps and x it ∈ R M is M observations for the corresponding time step.
Our aim is to build a classifier that generalizes well over target samples T by leveraging the labeled source samples S. Importantly, labels for the target domain are not available during training. Instead, we later use the labeled target samples ∼ D T only for the purpose of evaluating our framework.
The above setting is directly relevant for practice [15,22,27,57]. For example, medical time series from different health institutions differ in terms of patient cohorts, medical routines, reporting practice, etc., and, therefore, are subject to substantial domain shifts. As such, data from training and data from deployment should be considered as different domains. Hence, in order to apply machine learning for risk scoring or other medical use cases, it is often helpful to adapt the machine learning model trained on one domain S for another domain T before deployment.

Proposed CLUDA Framework
In this section, we describe the components of our framework to learn domain-invariant, contextual representation of time series. We start with an overview of the neural architecture, and then describe how we (1) perform adversarial training, (2) capture the contextual representation, and (3) align contextual representation across domains.

Architecture
The neural architecture of our CLUDA for unsupervised domain adaptation of time series is shown in Fig. 1. Specifically, the components are as follows: (1) The feature extractor network F (·) takes the original time series x s and x t from both domains and creates corresponding embeddings z s and z t , respectively. (2) The classifier network C(·) is trained to predict the label y s of time series from the source domain using the embeddings z s . (3) The discriminator network D(·) is trained to distinguish source embeddings z s from target embeddings z t . For such training, we introduce domain labels d = 0 for source instances and d = 1 for target instances.

Adversarial Training for Unsupervised Domain Adaptation
To train the above components, we minimize a combination of two losses: (1) Our prediction loss L c trains the feature extractor F (·) and the classifier C(·). We train both jointly in order to correctly predict the labels from the source domain. For this, we define the prediction loss where L pred (·, ·) is the cross-entropy loss.
(2) Our domain classification loss L disc is used to learn domain-invariant feature representations.
Here, we use adversarial learning [16]. To this end, the domain discriminator D(·) is trained to minimize the domain classification loss, whereas the feature extractor F (·) is trained to maximize the same loss simultaneously. This is achieved by the gradient reversal layer R(·) between F (·) and D(·), defined by Hence, we yield the domain classification loss

Capturing Contextual Representations
In our CLUDA, we capture a contextual representation of the time series in the embeddings z s and z t , and then align the contextual representations of the two domains for unsupervised domain adaptation. With this approach, we improve upon the earlier works in two ways: (1) We encourage our feature extractor F (·) to learn label-preserving information captured by the context. This observation was made earlier for unsupervised representation learning yet outside of our time series settings [2,8,9,17,19,46,47]. (2) We further hypothesize that discrepancy between the contextual representations of two domains is smaller than the discrepancy between their feature space, and therefore, the domain alignment task becomes easier.
To capture the contextual representations of time series for each domain, we leverage contrastive learning. CL is widely used in unsupervised representation learning for the downstream tasks in machine learning [8,9,21,31,39,55,58]. In plain words, CL approach aims to learn similar representations for two augmented views (positive pair) of the same sample in contrast to the views from other samples (negative pairs). This leads to a maximization of the mutual information between two views and, therefore, captures a contextual representation [2,46,47].
In our framework (see Fig. 2), we leverage contrastive learning in form of momentum contrast (MoCo) [21] in order to capture the contextual representations from each domain. Accordingly, we apply semantic-preserving augmentations [10,26,55] to each sample of multivariate time series twice. Specifically, in our framework, we sequentially apply the following functions with random instantiations: history crop, history cutout, channel dropout, and Gaussian noise (see Appendix C for details). After augmentation, we have two views of each sample, called query x q and key x k . These two views are then processed by the feature extractor to get their embeddings as z q = F (x q ) and z k =F (x k ). Here,F (·) is a momentum-updated feature extractor for MoCo.
To train the momentum-updated feature extractor, the gradients are not backpropagated throughF (·). Instead, the weights θF are updated by the momentum via where m ∈ [0, 1) is the momentum coefficient. The objective of MoCo-based contrastive learning is to project z q via a projector network Q(·) and bring the projection Q(z q ) closer to its positive sample z k (as opposed to negative samples stored in queue {z kj } J j=1 ). After each training step, the batch of z k 's are stored in queue of size J. As a result, for each domain, we have the following contrastive loss where τ > 0 is the temperature scaling parameter, and where all embeddings are normalized. Since we have two domains (i.e., source and target), we also have two contrastive loss components given by L s CL and L t CL and two queues given by queue s and queue t , respectively. Fig. 2 shows some network components for source and target samples twice for better readability, even though the weights of F (·),F (·), and Q(·) are actually shared across domains. We further plot the discriminator D(·) separately to avoid overlapping arrows in the figure.

Aligning the Contextual Representation Across Domains
Our CLUDA framework further aligns the contextual representation across the source and target domains. To do so, we build upon ideas for nearest-neighbor contrastive learning [12] from unsupervised representation learning, yet outside of our time series setting. To the best of our knowledge, ours is the first nearest-neighbor contrastive learning approach for unsupervised domain adaptation of time series.
In our CLUDA framework, nearest-neighbor contrastive learning should facilitate the classifier C(·) to make accurate predictions for the target domain. We achieve this by creating positive pairs between domains, whereby we explicitly align the representations across domains. For this, we pair z t qi with the nearest-neighbor of z t ki from the source domain, denoted as NN s (z t ki ). We thus introduce our nearest-neighbor contrastive learning loss where NN s (·) retrieves the nearest-neighbor of an embedding from the source queries {z s qi } Ns i=1 .

Training
Overall loss: Overall, our CLUDA framework minimizes where hyperparameters λ disc , λ CL , and λ NNCL control the contribution of each component.

Implementation:
We implement the feature extractor F (·) via a temporal convolutional network (TCN) [3]. We configure its kernel size, dilation factor and number of layers to cover the entire history of the time series for each dataset. We implement the discriminator D(·), the classifier C(·), and the projector Q(·) via multilayer perceptrons with one hidden-layer. Appendix C provides all details of our architecture search for each component.

Experimental Setup
Our evaluation is two-fold: (1) We conduct extensive experiments using established benchmark datasets, which have been used in the earlier works of UDA on time series [51,52,5,29]. Thereby, we show how our CLUDA leads to increasing accuracy on target domains by an important margin.
(2) We show the applicability of our framework in a real-world setting with medical datasets. (

Prediction Performance on Medical Datasets
We further demonstrate that our CLUDA achieves state-of-the-art UDA performance using the medical datasets. For this, Table 1 lists 12 UDA scenarios created from the MIMIC dataset. In 9 out of 12 domain UDA scenarios, CLUDA yields the best mortality prediction in the target domain, consistently outperforming the UDA baselines. When averaging over all scenarios, our CLUDA improves over the performance of the best baseline (AdvSKM) by 2.2 % (AUROC 0.773 vs. 0.756).
Appendix H reports the ablation study for this experimental setup with similar findings as above.
Appendix G repeats the experiments for another medical dataset: AmsterdamUMCdb [45]. Again, our CLUDA achieves state-of-the-art performance.

Case Study: Application to Medical Practice
We now provide a case study showing the application of our framework to medical practice. Here, we evaluate the domain adaptation between two health institutions. We intentionally chose this setting as medical applications are known to suffer from a substantial domain shifts (e. g., due to different patient cohorts, medical routines, reporting practices, etc., across health institutions) [15,32,57]. We treat MIMIC and AmsterdamUMCdb (AUMC) as separate domains and then predict health outcomes analogous to earlier works [5,6,18,34,36]: decompensation, mortality, and length of stay (see Table  2). All details regarding the medical datasets (i. e., summary statistics, measurements, pre-processing etc.) and task definitions are given in Appendix B.  For better comparability in practice, we focus on the "performance gap": we interpret the performance from source → source setting as a loose upper bound. 2 We then report how much of the performance gap between no domain adaptation (w/o UDA) vs. the loose upper bound is filled by each method. Importantly, our CLUDA consistently outperforms the state-of-the-art baselines. For instance, in decompensation prediction for AUMC, our CLUDA (AUROC 0.791) fills 47.6 % of the performance gap between no domain adaptation (AUROC 0.771) and loose upper bound from the source → source setting (AUROC 0.813). In contrast, the best baseline model of this task (HoMM) can only fill 16.7 % (AUROC 0.778). Altogether, this demonstrates the effectiveness of our proposed framework.
Appendix I reports the detailed results for different performance metrics. Appendix J provides an ablation study. Both support our above findings.

Conclusion
In this paper, we propose a novel framework for unsupervised domain adaptation (UDA) of time series based on contrastive learning, called CLUDA. To the best of our knowledge, CLUDA is the first approach that learns domain-invariant, contextual representation in multivariate time series for UDA. Further, CLUDA achieves state-of-the-art performance for UDA of time series. Importantly, our two novel components -i.e., our custom CL and our NNCL -yield clear performance improvements. Finally we expect our framework of direct practical value for medical applications where risk scores should be transferred across populations or institutions.
[23] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu.  [8]. It maximizes the agreement between the embeddings of the two augmented views of the same sample and treats all the other samples in the same batch as negative samples. Nearest-neighbor CL (NNCL) [12] creates positive pairs from other samples in the dataset. For this, it takes the embedding of first augmented view and finds its nearest neighbor from the support set. Moment contrast (MoCo) [21] refers to the embeddings of two augmented views as query and key, and then constructs positive pairs for the sample as follows: Key embeddings are generated by a momentum encoder and stored in a queue (whose size is larger than the batch size), while all key embeddings are further used to construct negative pairs for the other sample. Thereby, MoCo generates more negative pairs than the batch size as compared to SimCLR, which is often more efficient in practice.
Contrastive learning for time series: Contrastive learning has been used for time series to learn contextual representation of time series in unsupervised settings. As a result, several methods emerged: Scalable representation learning (SRL) [14], neighborhood contrastive learning (NCL) [55], TS2Vec [56], and temporal neighborhood coding (TNC) [48] treat the neighboring windows of the time series as positive pairs and use other windows to construct negative pairs. For this, SRL, NCL, and TS2Vec minimize the triplet loss, contrastive loss, and hierarchical contrastive loss, respectively, while TNC trains a discriminator network to predict neighborhood information.
There are also more specialized methods. For example, contrastive learning of cardiac signals (CLOCKS) [26] leverages spatial invariance and constructs positive pairs from measurements of the different sensors of the same subject. Temporal and contextual contrasting (TS-TCC) [13] is a variant of CPC and maximizes the agreement between strong and weak augmentations of the same sample in an autoregressive model. Bilinear temporal-spectral fusion (BTSF) [54] constructs the positive pairs via dropout layer applied to the same sample twice and it minimizes a triplet loss for the temporal and spectral features. Time-Frequency Consistency (TF-C) [58] maximizes the agreement between time and frequency embeddings of the same sample.

B Dataset Details B.1 Benchmark Datasets
We select three sensor datasets that are most commonly used in the earlier works. In each dataset, participants perform various actions while wearing smartphone and/or smartwatches. Based on the sensor measurements, the task is to predict which action the participant is performing. Table 3 provides summary statistics for all datasets. Below, we provide additional information about each dataset.
WISDM. The dataset contains 3-axis accelerometer measurements from 30 participants. The measurements are collected at 20 Hz, and we use non-overlapping segments of 128 time steps to predict the type of the activity of a participant. There are 6 types of activities: walking, jogging, sitting, standing, walking upstairs and walking downstairs. This dataset is particularly challenging due to class imbalance across participants, i. e., there are some participants who did not perform all the activities.
HAR. The dataset contains the measurements of 3-axis accelerometer, 3-axis gyroscope, and 3-axis body acceleration from 30 participants. The measurements are collected at 50 Hz, and we use non-overlapping segments of 128 time steps to predict the type of the activity of a participant. There are 6 types of activities: walking, walking upstairs, walking downstairs, sitting, standing, and lying down.
HHAR. The dataset contains 3-axis accelerometer measurements from 30 participants. The measurements are collected at 50 Hz, and we use non-overlapping segments of 128 time steps to predict the type of the activity of a participant. There are 6 types of activities: biking, sitting, standing, walking, walking upstairs, and walking downstairs.  Table 4 shows the summary statistics of both medical datasets MIMIC and AUMC. Table 5 provides additional details for both datasets MIMIC and AUMC. Both comprise of 41 separate time series, which are then used to predict the outcomes of interest -i.e., decompensation, mortality, and length of stay -via unsupervised domain adaptation. Table 6 shows the number of patients and the number of samples for each split and each dataset. As a reminder, since we start making the prediction at four hours after the ICU admission, the same patient yields multiple samples when training/testing the models.

B.2 Medical Datasets
Pre-processing: We split the patients of each dataset into 3 parts for training/validation/testing (ratio: 70/15/15). We used a stratified split based on the mortality label. We proceeded analogous to previous works for pre-processing [5,6,18,20,34,36,37,55]. Each measurement was sampled to hourly resolution, and missing measurements were filled with forward-filling imputation. We applied standard scaling to each measurement based on the statistics from training set. The remaining missing measurements were filled with zero, which corresponds to mean imputation after scaling. We followed best practices in benchmarking data from intensive care units [20,37]. That is, for each of   the tasks, we start making the prediction at four hours after the ICU admission. In all our experiments, we used a maximum history length T = 48 hours. Shorter sequences were pre-padded by zero.

Tasks:
We compare the performance of our framework across 3 different standard tasks from the literature [5,6,18,20,34,36,37]. (1) Decompensation prediction refers to predicting whether the patient dies within the next 24 hours. (2) Mortality prediction refers to predicting whether the patient dies during his/her ICU stay. (3) Length of stay prediction refers to predicting the remaining hours of ICU stay for the given patient. This serves as a proxy of the overall health outcome. The distribution of remaining length of ICU stay contains a heavy tail (see Appendix A), which makes it challenging to model it as a regression task. Therefore, we follow the previous works [20,37] and divide the range of values into 10 buckets and perform an ordinal multiclass classification.
For each task, we performed unsupervised domain adaptation in both ways: MIMIC (source) → AUMC (target), and AUMC (source) → MIMIC (target). Later, we also report the corresponding performance on the test samples from the source dataset (i.e., MIMIC → MIMIC, and AUMC → AUMC). This way, we aim to provide insights to what extent the different UDA methods provide a trade-off for the performance in the source vs. target domain. It also be loosely interpreted as a upper bound for the prediction performance.
Performance metrics: We report the following performance metrics. The tasks for predicting (1) decompensation and (2) mortality are binary classification problems. For these tasks, we compare the area under the receiver operating characteristics curve (AUROC) and area under the precisionrecall curve (AUPRC). Results for AUPRC are in the Appendix C due to space limitation. The task of predicting (3) length of stay is an ordinal multiclass classification problem. For this, we report Cohen's linear weighted kappa, which measures the correlation between the predicted and ground-truth classes.
Summary statistics for "length of stay" Here, we provide additional summary statistics for the distribution of "length of stay". Figure 6 and Figure 7 show the length of stay distribution of all patients in the MIMIC and AUMC datasets, respectively. Further, Figure 8 and Figure 9 show the remaining length of stay distribution for all samples (i. e., all time windows considered for all patients) in MIMIC and AUMC, respectively. Recall that we divide the values of remaining length of stay into 10 buckets; the corresponding fraction of samples belonging to each bucket is reported in Figure 10.     The buckets are the following: one bucket for less than one day, one bucket each for days 1 through 7, one bucket for the interval between 7 and 14 days, and one bucket for more than 14 days.

C Training Details
In this section, we provide details on the hyperparameters tuning. Table 7 lists the tuning range of all hyperparameters. To avoid repetition, we list hyperparameters that appear at all methods in the first rows of Table 7. For each dataset (benchmark or medical) and each task (i. e., decompensation, mortality, and length of stay prediction), we performed a grid search for hyperparameter tuning separately for each method.
We implemented our CLUDA framework and all the baseline methods in PyTorch. For training and testing, we used NVIDIA GeForce GTX 1080 Ti with 11GB GPU memory. We minimize the loss of each method via Adam optimizer.
We implement the feature extractor F (·) via a temporal convolutional network (TCN) [3]. We set its kernel size 3 and dilation factor 2. For benchmark datasets, we use 6 layers with 16 channels, whereas for medical datasets, we use 5 layers with 64 channels. This configuration remains the same across all methods so that the difference in prediction performance is attributed to their novel UDA approach.
We now explain how we decide the search range of the hyperparameters (e. g., learning rate, weight decay). The low learning rate is preferred so that the methods converge to a certain loss after seeing all samples from each dataset. Especially, the medical dataset MIMIC has roughly 2.4M samples, and it requires ∼1.2K steps to iterate over all these samples with a batch size of 2048. With higher learning rates, the methods converge to a loss even before one iteration over the dataset. We observed that this leads to suboptimal prediction performance (i.e., lower AUROC, AUPRC and KAPPA scores). For the hyperparameters regarding the contrastive learning framework, we are informed by the configuration of MoCo [21] as a starting point. We explored a certain range to improve the performance. For the feature extractor F (·) and the classifier C(·), we used the best hyperparameter configuration obtained by w/o UDA as a starting point. For benchmark datasets, we trained all methods for max. 5,000 training steps with a batch size of 128. For medical datasets, we trained all methods for max. 30,000 training steps with a batch size of 2048 (except AdvSKM, DDC, DSAN, and MMDA with a batch size of 1024 to fit into GPU). We applied early stopping based on the method performance on validation set without the labels from the target domain. To report the test performance of each method with the error bars, we repeated each experiment with 10 different random seeds (i. e., 10 different random initializations).
Here, we compare the runtimes of each method. For this, we use MIMIC-III (the largest dataset in our experiments). We report average runtimes per 100 training steps since the total runtime (i. e., total number of training steps) varies with the step of early stopping applied at each run. For each method, the average runtimes (per 100 training steps) are the following: 44

Augmentations
To capture the contextual representation of medical time series, we apply semantic-preserving augmentations [10,26,55] in our CLUDA framework. We list the augmentations and their optimal hyperparameters (search range in parenthesis) below: History crop: We mask out a minimum 20 % (10 % -40 %) of the initial time series with 50 % (20 % -50 %) probability.
Gaussian noise: We apply Gaussian noise to each measurement independently with standard deviation of 0.1 (0.05 -0.2).
We apply these augmentations sequentially to each time series twice. As a result, we have two semantic-preserving augmented views of the same time series for our CLUDA framework. Of note, we trained all the baseline methods with and without the augmentations of time series. We always report the their best results.

D UDA on Benchmark Datasets
We perform the activity prediction as a UDA task based on the benchmark datasets WISDM, HAR, and HHAR. For each dataset, we present the prediction results for 10 randomly selected source-target pairs. For each source-target pair, we repeat the experiments with 10 random initializations and report the mean values. Table 8 shows the accuracy on the target domains and average accuracy for each dataset. Similarly, Table 9 shows the Macro-F1 on the target domains and average Macro-F1 for each dataset. Overall, our CLUDA outperforms the UDA baselines by a large margin, as discussed in the main paper. Specifically, CLUDA achieves the best accuracy in 28 out of 30 UDA scenarios and the best Macro-F1 in 27 out of 30 UDA scenarios. Thereby, the results confirm the effectiveness of our method.

E Embedding Visualization
In this section, we provide the t-SNE visualization (see Fig. 11) for the embeddings of each method from HHAR dataset. When there is no domain adaptation (see Fig. 11a

F Ablation Study for UDA on Benchmark Datasets
We further conduct an ablation study on the benchmark datasets WISDM, HAR, and HHAR. We use the same variants of CLUDA from the main paper (see Sec. 6.1: w/o CL and w/o NNCL, w/o CL, w/o NNCL, and w/o Discriminator. Similar to the main experiments, for each dataset, we present the prediction results for 10 randomly selected source-target pairs. For each source-target pair, we repeat the experiments with 10 random initializations and report the mean values. Table 10 shows the accuracy on the target domains and average accuracy for each dataset. Similarly, Table 11 shows the Macro-F1 on the target domains and average Macro-F1 for each dataset. Overall, our complete CLUDA outperforms all its variants by a significant margin, which confirms our chosen architecture.

G UDA across Various Age Groups
Following the earlier works [36, 5], we conducted extensive experiments to compare the UDA performance of our CLUDA framework across various age groups. We consider the following groups: (1) Group 1: working-age adult (20 to 45 years old patients); (2) Group 2: old working-age adult (46 to 65 years old patients); (3) Group 3: elderly (66 to 85 years old patients); and (4) Group 4: seniors (85+ years old patients). Therefore, within each dataset (MIMIC and AUMC), we list the results of all combinations of Source → Target for mortality prediction (i. e., Group 1 → Group 2, Group 1 → Group 3, . . . , Group 4 → Group 3). Results are shown in Table 1 (in the main paper) for MIMIC and Table 12 for AUMC.
We further extend the experiments to across datasets. That means, we pick the source domain as one age group from one dataset (e. g., Group 1 of MIMIC) and pick the target domain as one age group from the other dataset (e. g., Group 3 of AUMC). We, again, conducted the experiments for all combinations of age groups across the datasets. Results are shown in Table 13 from MIMIC to AUMC and Table 14 from AUMC to MIMIC.
We report the mean over 10 random initialization. For better readability, we omitted the standard deviation. Nevertheless, we highlight performance results in bold when corresponding baselines are outperformed at a significant level.     In total, our ablation study counts 56 new experiments. We report the mean over 10 random initialization. For better readability, we omitted the standard deviation. Nevertheless, we highlight performance results in bold when corresponding baselines are outperformed at a significant level.
We make the following important findings. First, our CLUDA works overall best on the target domain, thereby justifying our chosen architecture. Second, the models w/o CL and w/o NNCL perform significantly worse than our complete framework, which justifies our choice for incorporating both components. Third, we compare w/o Discriminator and our CLUDA. As demonstrated by our results, the discriminator is consistently responsible for better UDA for the target domain consistently. Overall, its performance improvement is significant but the gain is smaller than the other components.

I Prediction Results of Practical Use Case
The main paper reported the average UDA performance between MIMIC and AUMC without the standard deviation of the results. Here, we provide the full results with gap filled (%) calculated for each method and additional AUPRC metric for decompensation and mortality predictions. Table 19 and Table 20 show the decompensation prediction results. Table 21 and Table 22 show the mortality prediction results. Table 23 show the length of stay prediction results.   The results confirm our findings from the main paper: overall, our CLUDA achieves the best performance in both source and target domains.

J Ablation Study (Additional Results)
Here, we additionally provide our ablation study for the case study presented in Sec. 6.3. Specifically,    Overall, the ablation study with different variants of our CLUDA confirms the importance of each component in our framework. Specifically, our CLUDA improves the prediction performance over all of its variants in all tasks except one (mortality prediction from MIMIC to AUMC). For this task, it is important to note that the best performing variant is w/o Discriminator, which has all the novel components of our CLUDA framework.

K CLUDA with SimCLR
In our CLUDA framework, we capture contextual representation in time series data by leveraging contrastive learning. Specifically, we adapt momentum contrast (MoCo) [21] for contrastive learning in our framework. This choice is motivated by earlier research from other domains [21,9,55,12], where MoCo was found to yield more stable negative samples (due to the momentum-updated feature extractor) as compared to other approaches throughout each training step, such as SimCLR [8].
In principle, stability yields stronger negative samples for the contrastive learning objectives and, therefore, increases the mutual information between the positive pair (i. e., two augmented views of the same sample). Furthermore, MoCo allows storing the negative samples within a queue, facilitating a larger number of negative samples for the contrastive loss as compared to SimCLR. As shown earlier [2,46,47], the lower bound of the mutual information between the positive pair increases with a larger number of negative samples in CL. With that motivation, we opted for MoCo [21] in our CLUDA instead of SimCLR [8]. Nevertheless, we evaluate our choice through numerical experiments below.
We now further perform an ablation study where we repeat the experiments with SimCLR (instead of MoCo) for our case study from Sec. 6.3. Specifically, we provide results for decompensation prediction (see Table 30), mortality prediction (see Table 31), and length of stay prediction (see Table 32).   The results confirm our choice for MoCo instead of SimCLR in capturing the contextual representation in time series. Specifically, our CLUDA improves the result of CLUDA w/ SimCLR in all tasks by a large margin. Despite being inferior to our CLUDA, CLUDA w/ SimCLR achieves better UDA performance compared to other baseline methods in decompensation prediction from MIMIC to AUMC, mortality prediction from AUMC to MIMIC, and length of stay prediction from MIMIC to AUMC. This shows the importance of leveraging the contextual representation into unsupervised domain adaptation. Besides, it highlights that our CLUDA can be further improved in the future with the recent advances in capturing the contextual representation of time series.