{'fraction_non_alphanumeric': 0.03512987281991797, 'fraction_numerical': 0.010066697046273665, 'mean_word_length': 4.550701310646126, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Gradual Domain Adaptation via Normalizing Flows

Conventional domain adaptation methods do not work well when a large gap exists between the source and the target domain. Gradual domain adaptation is one of the approaches to address the problem by leveraging the intermediate domain, which gradually shifts from the source to the target domain. The previous work assumed that the number of the intermediate domains is large and the distance of the adjacent domains is small; hence, the gradual domain adaptation algorithm by self-training with unlabeled datasets was applicable. In practice, however, gradual self-training will fail because the number of the intermediate domains is limited, and the distance of the adjacent domains is large. We propose using normalizing flows to mitigate this problem while maintaining the framework of unsupervised domain adaptation. We generate pseudo intermediate domains from normalizing flows and then use them for gradual domain adaptation. We evaluate our method by experiments with real-world datasets and confirm that our proposed method mitigates the above explained problem and improves the classification performance.


Introduction
In a standard problem of learning predictive models, it is assumed that the probability distributions of the test and the training data are the same. The prediction performance generally deteriorates when the above assumption does not hold. The simplest solution is discarding the training data and collecting new samples from the distribution of the test data. However, the simplest solution is inefficient and sometimes impossible, and there is a strong demand for utilizing the valuable labeled data in the source domain.
Transfer learning methods do not assume that the probability distributions of the test and the training data are the same, and can utilize the valuable labeled data effectively. Domain adaptation [3] is one of the transfer learning frameworks in which the probability distributions of the prediction target data and the training data are different, but the task of machine learning is the same. In domain adaptation, the source domain means a distribution with a lot of labeled samples, and the target domain means a distribution with a few or no labeled samples. The case where there are no labels from the target domain is called unsupervised domain adaptation and has been the subject of much research, including theoretical analysis and real-world application [3,7,30,39,48].
In domain adaptation, the predictive power of the target data deteriorates when the "distance 1 " between the source and the target domain is large. Kumar et al. [24] proposed gradual domain adaptation, which assumed that even when the distance between the source and the target domain is large, the shift from the source to the target domain occurs gradually and the unlabeled datasets from intermediate domains are available. In gradual domain adaptation, it is possible to predict data in the target domain far from the source domain since the intermediate domains connect the source and the target domain. The intermediate domains gradually shift from the source to the target domain, and they densely interpolate between the source and the target domain.
The key assumption of the gradual domain adaptation [24] is that there are many indexed intermediate domains, and they connect the source and the target domain densely. Hence, gradual self-training was possible without the need for labeled data. In practice, however, the number of the intermediate domain is limited. Therefore, the distances of the adjacent domains are large, and gradual self-training does not work well. Sagawa and Hino [40] tackled this problem by using a multifidelity active learning algorithm and requesting a small number of queries from each domain. However, the method assumed that we have access to the intermediate and the target labels. In this paper, we propose a method that does not require additional labels and mitigates the problem that distances of the adjacent domains are large.
Our key idea is to use generative models to learn gradual shifts between domains. We use samples from generative models as data from unobserved intermediate domains, and interpolate the initially given domains to prevent the failure of gradual self-training. Figure 1 shows the schematic diagram of the proposed method. We focus on the normalizing flow (NF) [35] as a generative model to describe gradual shifts. A normalizing flow learns a transformation between an unknown probability distribution and a known probability distribution. We assume that the shifts in the transformation process correspond to gradual shifts between the source and the target domain.
The rest of the paper is organized as follows. We review related works in Section 2. Then we review the gradual domain adaptation algorithm, an important previous work, in Section 3. We introduce our proposed method in Section 4. In Section 5, we present the experimental results. We discuss the applicability of our method and conclusion in Section 6.

Related works 2.1 Gradual domain adaptation
In conventional domain adaptation, several methods have been proposed to improve the prediction performance on the target data by using a single intermediate domain [6,8,9,12,14,18]. In these methods, generative models are used to obtain the intermediate data that consist of mixture of the source and the target data. These methods can be regarded as learning invariant features between the source and the target domain. In addition, Wang et al. [43] assumed the problem where there are multiple source domains with indices. It corresponds to the gradual domain adaptation problem where we can access all labels of the intermediate domains.
Kumar et al. [24] proposed gradual domain adaptation, and they showed that by self-training with unlabeled datasets, it is possible to adapt to a large domain gap. It

Normalizing flows
A normalizing flow learns an invertible transformation between an unknown distribution and a parametric distribution, and it is broadly classified into discrete and continuous types. The primal research problem of discrete type normalizing flows is to improve the expressive power of flow-based models for a given data set. Several methods have been proposed as discrete type normalizing flows (RealNVP [11], Glow [22], Flow++ [17]). Grathwohl et al. [15] proposed FFJORD as a continuous normalizing flow (CNF), followed by the studies to improve the computational efficiency [19,33], and on the representation on Riemannian manifolds [31]. Normalizing flows have been applied to several specific tasks, for example, data generation (images [28], 3D point clouds [37], chemical graphs [26]), anomaly detection [23], and semi-supervised learning [21]. In several methods [20,29,45], NFs are used to compensate for other generative models like generative adversarial networks [34] and variational autoencoders [46].
Normalizing flows for dynamical system [38], time series [4], and video [25] data are also developed. Although these methods can deal with sequential data structure, it is not designed for dealing with multiple domain structure. Whang et al. [44] proposed the NF for inverse problems. In the presence of trained flow, they propose to add another flow for inverse problems. The method can be applied to a variety of tasks, for example, image inpainting, super-resolution, and compressed sensing. They empirically showed that a face image with the lower half painted black could be restored. However, this method needs the transformation rule such that the lower half of the images are hidden. If the rules of domain shifts were simple and known, we can use the this method.
However, in general, the rules of domain shifts were unknown.
Domain adaptation methods with NFs and adversarial training are also developed [10,16]. These methods individually train two normalizing flows for the source and the target domain, and execute domain alignment in a common latent space using adversarial discriminators. We can extend these methods to learn multiple intermediate domains together, but it will cause instability of training because these methods involve adversarial training. Usman et al. [41] proposed a method using log-likelihood ratio statistics in the context of distribution alignment to mitigate an instability of adversarial training. However, there are still some technical issues with the handling of adversarial training.

Background: gradual domain adaptation
We first review the details of the gradual domain adaptation method proposed by Kumar et al. [24] following a concise description by Chen and Chao [5]. Then we confirm that the predictive power deteriorates when the distance of the adjacent domains is large.
We consider a multiclass classification problem. Let X ∈ X = R d and Y ∈ Y = {1, 2, . . . , C} be random variables of the input and label, respectively, and let x and y be their realizations. The source dataset has labels S = {(x

whereas the intermediate datasets and the target dataset do not have labels U
X,Y for a pair of variables X and Y . The source domain corresponds to j = 1, and the target domain corresponds to j = K. When j is small, the domain is considered to be similar to the source domain. On the contrary, when j is large, the domain is considered to be similar to the target domain. The similarity of two domains is measured by maximum per-class Wasserstein-infinity distance ρ defined in [42]. In this study, the index j is not a discrete but a continuous variable since we use CNF to interpolate the domains. We describe the details of interpolation in Section 4.2. Let T = {j | j ∈ [1, K]} be the set of indices in the initially given domains.
The gradual domain adaptation algorithm consists of two steps. Firstly, we train the base model with the source dataset. Then, we sequentially apply self-training to the base model with unlabeled datasets from adjacent domains. Let h θ be a classifier parameterized by θ ∈ Θ, and its output be a conditional probability . . .
Depending on the context, θ indicates the classifier h θ . Let : Y × Y → R ≥0 be a loss function, and consider training the base model θ (1) by minimizing the loss on the source dataset For the joint distribution p X,Y , we define the expected loss with parameter θ as follows The previous work [24] showed that a hard label is more suitable than a soft label for pseudo-labels during self-training of gradual domain adaptation. We define the sharpen function to obtain a hard label from the output of a classifier h θ sharpen(h θ (x)) = argmax c∈{1,...,C} Pr(y = c|x, θ).
We make predictions on the unlabeled dataset in the next domain with the classifier of the current domain θ t . Let ST(θ t , U ) a function that returns the self-trained model parameter of the predictive model for x ∈ U by inputting the current model parameter θ t and an unlabeled dataset U . We only apply self-training between adjacent domains. We train the new model by minimizing the loss on the unlabeled dataset ST(θ t , U ) = argmin The output of gradual domain adaptation is the classifier for the target domain h θ (K) . The parameter θ (K) is obtained by applying self-training to the base model h θ (1) in the sequence of unlabeled datasets U (2) , U (3) , . . . , U (K) : Kumar et al. [24] made the following three assumptions and conducted a theoretical analysis.
1. Low loss assumption: In all domains, there is a classifier with low expected loss L r (θ, p X,Y ), and their L 2 norm is less than or equal to the constant R ∈ R >0 , θ 2 ≤ R.
2. Gradual shift assumption: The maximum per-class Wasserstein-infinity distance [42] between any pair of adjacent domains is ρ ≤ 1 R , and there is no label shift between any pair of adjacent domains. 3. Bounded model complexity: In all domains, the Rademacher complexity of a classifier is bounded by B √ n j , where B ∈ R >0 is a constant and n j is the number of samples of each domain.
The upper bound of the expected loss of the target classifier obeys the following inequality with at least probability δ ∈ (0, 1] (5) Since we consider the problem where the number of accessible intermediate domains is limited, it is natural that the second assumption is hard to hold. When the distance of the adjacent domain ρ is large, Eq. (5) suggests that the bound for the expected loss of the target classifier would become loose. To tackle the problem where the distance of the adjacent domains is large, we propose a method utilizing generative models.

Proposed method 4.1 Interpolation of the domains by normalizing flows
The limited number of intermediate domains would make the distance of the adjacent domains ρ large. For example, consider U (2) and U (3) as the given unlabeled datasets of adjacent domains. If we have datasets like U (2.1) , U (2.5) , and U (2.8) , we can interpolate between U (2) and U (3) . The sequence like U (2) , U (2.1) , U (2.5) , U (2.8) , and U (3) make the distance of the adjacent domains ρ small. We propose a method with CNFs that generate interpolating datasets such as U (2.1) , U (2.5) , and U (2.8) .
A normalizing flow transforms an unknown probability distribution into a known probability distribution, such as the standard normal distribution. Since an invertible function is used for NF, we can also use NF as a generative model. Let z ∈ R d be a datum from the standard normal distribution p Z , and we aim to learn the continuous change x (K) → x (K−1) · · · → x (1) → z by CNFs [15]. We introduce a time index t ∈ R + to represent continuous changes, and the index of each domain j is considered as a time point. If we apply gradual self-training with the initially given domains, the prediction performance on the target data is low because the distances of adjacent domains ρ will be large. We prepare some indices for interpolating the initially given domains and they are identified with time indices. We describe the details of the interpolation in Section 4.2. Let g and v be functions for variable transformation. The function g : R d × R + → R d outputs latent variable according to t. We set t = 0 for z, z = g(z, 0). Let v be an arbitrary neural network parameterized by ω and represent the change of g along t. We express an ordinary differential equation (ODE) by a neural network v as dg dt = v(g(·, t), t; ω).
In CNFs, the amount of change in g is obtained by integrating v with respect to t. The integrated value of v is used for variable transformation, from x to z and z to x: x (j) = g(z, j) = g(z, 0) + j 0 v(g(z, t), t; ω)dt.
We note that x (j) = g(x (j) , j). Up to this point, we have introduced variable transformation by CNFs. Next, we consider the Jacobian in CNF. While NF requires a computation of the Jacobian, in CNF, we compute the equivalent part of the Jacobian by integrating the time derivative of the log-likelihood. The time derivative of the log-likelihood is given by Note that ∂v/∂g is a Jacobian of v, and ∂v k /∂g l implies partial differentiation of the kth elemtnt of v by the l-th element of g. Computation of Jacobian for high-dimensional variables is computationally prohibitive. Therefore, in this study, we use a dimension reduction method as a preprocessing to reduce the computational cost. We describe the details of the preprocessing in Section 5. To summarize the variable transformation and the computation of the Jacobian, CNF is formulated as a problem of maximizing the following log-likelihood: where g(x (j) , j) = x (j) and d dt log p(g) = 0 are initial values. We use these CNFs to interpolate the initially given domains. Namely, to realize the interpolation, we maximize the log-likelihood of our flow-based model on all the data from initially given domains. We train our flow-based model by minimizing the following loss

Gradual self-training via normalizing flows
First, we explain how to interpolate the observed domains by using CNFs. Let T be a set of indices of the given domains, and the points in T are regarded as time points in the flow. We introduce α ∈ R + as a hyperparameter to tune the interval of interpolation. We define T gradual as a set of time indices corresponding to pseudo intermediate domains: To generate samples from the pseudo dataset with the index j ∈ T gradual , we sample data points {z i } n j i=1 from the standard normal distribution and convert the data along the flow defined by Eq. (7). We use the converted data as pseudo intermediate data.
In Figure 2, we show the example of the interpolation of existing domains with CNFs. We use the two-moon dataset as the source dataset and prepare an intermediate and the target dataset by rotating the source dataset. Visually, we can confirm that the interpolation of the domains is realized as expected. We note that in our preliminary experiments, we also tried the interpolation with discrete type normalizing flows, but the results were not as good as that with CNFs. Next, we consider the self-training with the datasets that includes the existing and the generated dataset. We train the base model with the source dataset by Eq. (1). Then, we apply gradual the self-training to the base model with the union of the existing unlabeled dataset and the generated unlabeled dataset. We show the details of our method in Algorithm 1.

Self-training cycle consistency
The proposed method requires us to set an appropriate number of the intermediate domains for gradual domain adaptation, which is controlled by the parameter α. Although we can interpolate the given domains by our proposed method, Eq. (5) suggests that the loss would increase as the number of self-training increases, so we have to avoid excessive interpolation and estimate an appropriate value of α. We propose a method to determine an appropriate value of α based on cycle consistency [5,27]. Concretely, we get the target classifier h θ (K) by gradual self-training and use the classifier to obtain the pseudo-labels from the target domain. Then the target pseudo-labels are used as correct labels. We apply gradual self-training from the target to the source domain again, and generate dataset, convert z to x by Eq. (7) 11: substitute the generated unlabeled dataset for U as U ← {g(z i , t)} T gradual ← T gradual \ {t} 16: end while 17: θ (K) ← θ t obtain the source classifier trained with pseudo-labels. Since we have the true labels from the source domain, we can compute the accuracy of the source classifier. Then, we train the new base model hθ (K) by minimizing the loss on the target dataset with pseudo-labelŝ Let be the source dataset excluding label information. We sequentially apply gradual self-training to U (K−1) , . . . , U (1) , and obtain the source classifierθ (1) : . . .
We select an appropriate α for our proposed method that gives the lowest cycle consistency loss (hθ (1) i ).

Experiments
We evaluate the performance of our proposed method on two real-world datasets. For our experiment, we use the implementation of CNF 2 , which is provided by Growth et al. [15]. We use Pytorch [36] to implement all procedures in our proposed method other than the procedure of CNF. We conduct all experiments on Amazon Web Services (AWS) [1] Elastic Compute Cloud (EC2) instance whose type is g4dn.8xlarge. Source code to reproduce the experimental results is available at https://github.com/ssgw320/gdacnf.

datasets
We use existing datasets with modifications for gradual domain adaptation. Since we are considering the situation that the distances of the adjacent domains are large, we prepare only one intermediate domain.
Rotating MNIST: We add rotations to the MNIST data. The rotation angle is 0 for the source, π/6 for the intermediate and π/3 for the target dataset.
Portraits: The Portraits dataset [13] includes photographs of American high school students from 1905 to 2013, and the task is gender classification. We sort the dataset in ascending order by year and split the dataset. The source dataset includes data from the 1900s to the 1930s. The intermediate dataset includes data from the 1940s and the 1950s. The target dataset includes data from the 1960s.

Effect of interpolation by normalizing flows
We train CNFs with three datasets from the source, the intermediate, and the target domain, that are prepared in Section 5.1. Due to the high computational cost for training CNFs, we apply UMAP [32] for reducing the dimension as a preprocessing. UMAP estimates the manifold structure in the original space and then seeks an appropriate embedding in an arbitrary low dimensional space. UMAP can also make use of partial labels as semi-supervised learning; hence we utilized the labels of the source dataset as the partial labels. Since we do not know the appropriate intrinsic dimension for embedding, we selected the dimension with which the predictive performance in the dimension-reduced source domain was the highest. All other parameters in UMAP are set to their default values. Note that our proposed method does not essentially require dimension reduction by UMAP, but it is used only to reduce the computational cost of training CNFs. Further improvement of computational efficiency of CNF will allow direct application of CNF without dimension reduction. One continuous normalizing flow block consists of 5 fully connected layers with 64 nodes in each layer. To improve the expressiveness, we connect three flow blocks and add one batch normalization layer between each block. The classifier consists of 2 fully connected layers with 32 nodes in each layer. We add one dropout and one batch normalization layer after the last fully connected layer of the classifier and set 0.001 as the value of weight decay.
We train CNFs with unlabeled dataset from the source, the target, and the one intermediate domain. We show the self-training results with various α values in Figure 3. Training of CNFs was done only once, and the self-training was repeated five times. In Rotating MNIST, we confirm an improvement of the predictive power for the target dataset (forward) and cycle consistency (backward) when α is sufficiently small. In Portraits, we confirm an improvement for forward, but there is no improvement in backward. We conjecture that a little mistake of pseudo-labeling makes the cycle consistency loss of Portraits large.  Figure 3: The hyperparameter α controls the interval of interpolation. When the hyperparameter α is sufficiently small, the given domains are well interpolated, and the predictive power for the target dataset is improved.
To verify its effectiveness, we compare our proposed method and the following baselines. Source only: We train a classifier with the source dataset only. w/o CNFs support: We apply gradual self-training with the initially given domains. w CNFs support: The proposed method. We interpolate the initially given domains by CNFs and apply gradual self-training. We confirm that our proposed method shows the best predictive power for both datasets.

Discussion & Conclusion
Gradual domain adaptation is one of the promising approaches to address the large domain gap problem by leveraging the intermediate domain. The previous work [24] assumed that the intermediate domain gradually shifts from the source to the target domain, and the distance of the adjacent domains is small. In thie study, we consider the problem where the distance of the adjacent domains is large. The proposed method mitigates the problem by utilizing normalizing flows. We evaluate the effectiveness of our proposed method on two real-world datasets. The hyperparameter α controls the interval of interpolation, and it is estimated by using the cycle consistency. However, there are some issues in handling the cycle consistency, and we conjecture that unreliable pseudo-labeling causes these issues. We note that in our preliminary experiments, we also tried the interpolation by using the CNFs, which trained without the In the future, we plan to address these problems and conduct a theoretical analysis to support the effectiveness of the use of the intermediate domains.