{'fraction_non_alphanumeric': 0.02994426136132471, 'fraction_numerical': 0.012653077830668456, 'mean_word_length': 4.662289794978116, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adaptive WGAN with loss change rate balancing

Optimizing the discriminator in Generative Adversarial Networks (GANs) to completion in the inner training loop is computationally prohibitive, and on finite datasets would result in overfitting. To address this, a common update strategy is to alternate between k optimization steps for the discriminator D and one optimization step for the generator G. This strategy is repeated in various GAN algorithms where k is selected empirically. In this paper, we show that this update strategy is not optimal in terms of accuracy and convergence speed, and propose a new update strategy for Wasserstein GANs (WGAN) and other GANs using the WGAN loss(e.g. WGAN-GP, Deblur GAN, and Super-resolution GAN). The proposed update strategy is based on a loss change ratio comparison of G and D. We demonstrate that the proposed strategy improves both convergence speed and accuracy.


Introduction
GANs [8] provide an effective deep neural network framework that can capture data distribution. GANS are modeled as a min-max two-player game between a discriminator network D ψ (x) and a generator network G θ (z). The optimization problem solved by GAN [21] is given by: where G : Z → X maps from the latent space Z to the input space X; D : X → R maps from the input space to a classification of the example as fake or real; and f : R → R is a concave function. In the remainder of this paper, we use the Wasserstein GAN [3] obtained when using f (x) = x.
GANs have been shown to perform well in various image generation applications such as: deblurring images [17], increasing the resolution of images [18], generating captions from images [5], and generating images from captions [23]. Training GANs may be difficult due to stability and convergence issues. To understand this consider the fact that GANs minimize a probabilistic divergence between real and fake (generated by the generator) data distributions [22]. Arjovsky et al. [2] showed that this divergence may be discontinuous with respect to the parameters of the generator, and may have infinite values if the real data distribution and the fake data distribution do not match.
In order to solve a divergence continuous problem, WGAN [3] uses the Wasserstein-divergence by removing the sigmoid function in the last layer of the discriminator and so restricting the discriminator to Lipschitz continuous functions instead of the Jensen-Shannon divergence in the original GAN [8]. WGAN will always converge when the discriminator is trained until convergence. However, in practice, WGAN is trained with a fixed number (five) of discriminator update steps for each generator update step.
Even though WGAN is more stable than the original GAN, Mescheder et al. [19] proved that WGAN trained with simultaneous or alternating gradient descent steps with a fixed number of discriminator updates per generator update and a fixed learning rate h > 0 does generally not converge to the Nash equilibrium for a Dirac-GAN, where the Dirac-GAN [19] is a simple prototypical GAN. Furthermore, in WGAN, the authors alternated between five optimization steps for the discriminator and one optimization step for the generator where there the ratio is set empirically. In this paper we address the number of training steps for the discriminator and generator and show how to adaptively change it during training so as to make the training converge faster to a more accurate solution.
The strategy we propose for balancing the training of the generator and discriminator is based on the discriminator and generator loss change ratios (r d and r g , respectively). Instead of a fixed update strategy we decide wheather to update the Grenerator or discriminator by comparing the weighted loss change ratios r d and λ·r g where the weight λ is a hyper-parameter assigning importance to r g . Using the weight λ it is possible to give preference to the update of one of the components thus, for example, training the discriminator more frequently as in the original WGAN. The reason for giving preference to one component has to do with the fact that the individual components affect the divergence reduction at different rates. Following Mescheder et al. [19] we demonstrate that the proposed strategy can reach a local convergence point for the Dirac-GAN problem, unlike the original update strategy which cannot achieve it.
To further demonstrate the advantage of the proposed update strategy, we train the WGAN [3], WGAN-GP [9], Deblur-GAN [17], and SR-WGAN [18] using the proposed strategy using different image datasets. These represent a wide range of GAN applications. Experimental results show that in general the proposed strategy converges faster while achieving in many cases better accuracy. An illustration is provided in Figure 1 where the proposed adaptive WGAN is compared with the traditional WGAN update strategy.
The main contribution of this paper is in proposing an adaptive update strategy for WGAN instead of the traditional fixed update strategy in which the update rate is set empirically. We show that the proposed strategy can reach local convergence for Dirac-GAN, unlike the traditional fixed update strategy which cannot do so. Experimental results on several problems with several datasets show that the proposed adaptive update strategy results in faster convergence and/or higher performance.

Related work
The question of which training methods for GANs actually converge was inversigated by Mescheder et al. [19] where they introduce the Dirac-GAN. The Dirac-GAN consists of a generator distribution p θ = δ θ and a linear discriminator D ψ (x) = ψ · x. In their paper they prove that a fixed point iteration F (x) is locally convergent to x, when the absolute values of the eigenvalues of the Jacobian F (x) are all smaller than 1. They further prove that for the Dirac-GAN, with both simultaneous and alternative gradient descent updates, the absolute values of the eigenvalues of the Jacobian F (x) in GANs with unregularized gradient descent (which include the original GAN, WGAN and WGAN-GP) are all larger or equal to 1, thus showing that these types of GANs are not necessarily locally convergent for the Dirac-GAN. To address this convergence issue, Mescheder et al. [19] added gradient penalties to the GAN loss and proved that regularized GAN with these gradient penalties can reach local convergence. This solution does not apply to WGAN and WGAN-GP which remain not locally convergent problems. Note that the WGAN is generally more stable and easier to train compared with GAN and hence the need for the adaptive update scheme we propose in this paper.
Heusel et al. [11] attempt to address the convergence problem in a different way by altering the learning rate. In their approach they use a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent using arbitrary GAN loss functions. Instead of empirically setting the same learning rate for both the generator and discriminator, TTUR uses different learning rates for them. This is done in order to address the problem of slow learning for regularized discriminators. They prove that training GANs with TTUR can converge to a stationary local Nash equilibrium under mild condition based on stochastic approximation theory. In their experiments on image generation, the show that WGAN-GP with TTUR gets better performance. Note however that empirically setting the learning rate is generally difficult and even more so when having to set two learning rates jointly. This makes applying this solution more difficult.
It is well understood that the complexity of the generator should be higher than that of the discriminator, a fact that makes GANs harder to train. Balancing the learning speed of the generator and discriminator is a fundamental problem. Unbalanced GANs [10] attempt to address this by pre-training the generator using variational autoencoder (VAE [15]), and using this pretrained generator to initialize the GAN weights during GAN training. An alternative solution is proposed in BE-GAN [4], where the authors introduce an equilibrium hyperparameter (E[f (−D(G(z)))]/E[f (D(x)]) to maintain the balance between the generator and discriminator. Training the two neural networks in this approach is time consuming and the equilibrium hyper-parameter is not suitable for all GAN training cases. For example it is not suitable when there is a content loss in the generator loss term L G as L G and the discriminator loss L D are not on the same scale.
A similar issue to the unbalanced training of the generator and discriminator in GANs arises in imbalanced training of multiple task networks. The GradNorm [6] approach provides a solution to balancing multitask network train-ing based on gradient magnitudes. In this approach, the authors multiply each of the single-task loss terms by weights, and automatically update those weights by computing a gradient normalization term. A relative inverse training rate (L current /L initial ) for each task is used to compute this normalization term. This strategy depends on a common loss term minimization where individual task terms are weighted and so is not suitable for GANs where there is no shared layer as in multi-task networks.

WGAN update strategy
In this section, we present our proposed update strategy to automatically set the update rate of the generator and discriminator instead of using a fixed rate as is commonly done. In WGAN or any GAN based on the WGAN loss, the Nash equilibrium is reached when the generator and discriminator loss terms stop changing. That is: where L c g , L c d represent the generator and discriminator loss in the current iteration respectively, and L p g , L p d represent the respective loss terms in the previous iteration. Since we play a min-max game in WGAN, it is crucial to balance the generator and discriminator losses. The loss terms L c g , L c d are given by: Comparing L c g , L c d directly to decide on an update policy is not possible because they are on different scales and so we define relative loss terms that can be compared. The relative loss terms are defined by computing the difference between the current and previous loss values and normalizing the difference by the loss magnitude. The relative change loss terms for the generator and discriminator are defined by: To prioritize the update of one component over the other as commonly done in GANs, we use an importance parameter λ. Thus, if r d > λ · r g , we update the discriminator, or otherwise update the generator. A larger loss change ratio of one component means that this component is in greater need for update. The details of our proposed adaptive WGAN are provided in Algorithm 1.

Convergence analysis
In this section, we demonstrate that with our proposed update strategy, WGAN can reach local convergence for
1: while θ has not converged do 2: P r a batch from the real data 3: Sample p(z) a batch of prior samples 4: if r d > λ · r g then # update the discriminator 5: else # update the generator 10: end if 13: if first iteration then 14: Dirac-GAN, whereas it cannot do so with a fixed update strategy. The GAN objective function is given by: The discriminator attempts to maximize this function whereas the generator attempts to minimize it. The goal is to find a Nash-equilibrium, where both components cannot improve their utility. The optimization is normally done using an alternating gradient descent where when training the generator the parameters are updated by: and when training the discriminator the parameters are updated by: The Dirac-GAN [19] consists of a generator distribution p θ = δ θ and a linear discriminator D ψ (x) = ψ · x. The true data distribution p D is given by a Dirac-distribution concentrated at 1. Thus, there is one parameter θ in the generator and one parameter ψ in the discriminator. For WGAN, we define f (t) = t and add a Lipschitz constraint (-0.5, 0.5) on the discriminator as in the original WGAN. Thus, the GAN objective function in Equation 5 is given by: The unique equilibrium point of the objective function in Thus, when training the generator, the parameters update in Equation 6 are given by: When training the discriminator, the parameters update in Equation 7 are given by: We employ our proposed update strategy using an alternating gradient descent based on Equations 10 and 11. We decide on the component to update by comparing the loss change ratios (r d and r g ) as described in Algorithm 1. The importance parameter λ is set to 1. For comparison we also apply the original WGAN update strategy of alternating gradient descent with fixed update steps (5 discriminator updates for each generator update). The results are shown in Figure 2. As can be observed in sub-figure (a) fixed WGAN updates (n d = 5, n g = 1) do not converge whereas in subfigure (b) adaptive updates following the proposed approach do converge to the Nash equilibrium point (1, 0).

Network architectures
To demonstrate our proposed adaptive training strategy as described in Algorithm 1 we evaluated several network architectures with and without adaptive training. Specifically, we evaluated standard WGAN [3] and WGAN-GP [9] networks for image synthesis, Deblur GAN [17] for image debluring using a Conditional Adversarial Network [20], and Super Resolution WGAN [13] for increasing image resolution using perceptual loss, content loss, and WGAN loss. Except for modifying the update strategy to become adaptive, we retained the original optimizer and loss functions in each of the evaluated networks, as can be found in the papers referenced above.
In our update strategy we set the importance parameter λ to different values. We observe that higher values of λ (up to 1000) perform better when the generation task is complex (e.g. in Deblur GAN and Super Resolution WGAN). Smaller values of λ (e.g. 1-5) work for the WGAN and WGAN-GP image generation networks. Increasing the value of λ results in training more the generator which is necessary due to the increased complexity of the generator. Note that training the generator more is in contrast to the suggestion in the original WGAN [3] paper where it is suggested to perform 5 training steps for the discriminator for each step of the generator. Experimental evaluation results are provided in the next section.

Experimental evaluation
In this section, we train different GANs with our updating strategy: WGAN, WGAN-GP, TTUR, Gradient Penalty, Deblur GAN and Super Resolution WGAN, and evaluate them both in quantitative and qualitative ways.

Experimental setup
We train both WGAN and adaptive WGAN for 500 epochs and using RmsProp optimizer (learning rate=0.00005 for both G and D). We train WGAN-GP and adpative WGAN-GP for 20k iterations and using Adam [14] optimizer (learning rate=0.0001 for both G and D, β 1 = 0.5, β 2 = 0.9). And both in original WGAN and WGAN-GP training, we follow the setting in [3] [9] that updates five times for D per updating one time for G. In adaptive WGAN and adaptive WGAN-GP training, we tried different λ: 1,3,5,10.
Meanwhile, we compare with the WGAN-GP TTUR (learning rate of G: lr g = 0.0001, learning rate of D: lr d = 0.0003 in TTUR; in adaptive WGAN-GP, lr g =0.0003 and lr d =0.0003 in order to keep in step), Ubalanced GAN, Gradient Penalty (we follow the hyper-parameters setting in original Gradident Penalty) and with TTUR (lr g =0.0001 and lr d =0.0003) and with our strategy (lr g =0.0003 and lr d =0.0003 in order to keep in step).  We train both Deblur GAN and adaptive Deblur GAN for 1500 epochs and using Adam optimizer (learning rate=0.0001 for both G and D, β 1 = 0.5, β 2 = 0.999). We train Deblur GAN 5 time on D and 1 time on G in each batch which followed the [17]. In adaptive Deblur GAN training, we tried λ: 1,10,100.
We train both Super Resolution WGAN and adaptive Super Resolution WGAN for 1000 epochs and using RmsProp optimizer (learning rate=0.001 for both G and D). We train Super Resolution WGAN one time on D and one time on G in each batch which followed the [13]. In adaptive Super Resolution WGAN training, we tried λ: 1,10,100,1000.

Datasets
To train the WGAN we use a 100 dimensional random noise vectors as input. For targets we use 3 datasets: the CIFAR- 10  To train the WGAN-GP and compare with the TTUR, Gradient Penalty, we all use the 128 dimensional random noise vectors as the input, and the CIFAR-10 dataset as the targets where the images are resized to 32x32 to match the original paper [9] setting.
To train the Deblur GAN we use the Caltech-UCSD Birds-200-2011 [26] dataset which has 200 classes of bird images with size 256x256. We synthesize blurred images from the original images using a sequence of six 3x3 Gaussian kernel convolutions. We then use the synthetic blurred images as the inputs and the corresponding original images as targets. To

Metrics
For evaluating image synthesis by WGAN and WGAN-GP we use as evaluation metrics the Inception Score (IS) [25] and Frenchet Inception Distance (FID) [7] which are well known, commonly used, GAN performance evaluation metrics. The Inception Score can measure a synthetic image quality by computing the expected KullbackLeibler divergence (KL divergence) between the marginal class distribution and conditional label distribution: where p(y|x) is the conditional label distribution of features extracted from the middle layers of the pretrained Inception-v3 model for generated images, and p(y) is the marginal class distribution. A higher IS value indicates better quality. The Frechet Inception Distance [7] that is given by: where F, G are two distributions of features extracted from the middle layers of a pretrained Inception-v3 model for generated and real images. The parameters µ x , µ y , Σ x , Σ y , are the mean vectors and covariance matrices of F and G. A lower FID score (the distribution F is similar to the distribution G) indicates better quality results. During training, in order to assess the convergence speed, we recorded each epoch's FID and saved the models in which epoch had the lowest FID value.
To evaluate the deblur and super resolution networks, we use two common metrics to measure the similarity between generated (deblurred or increased resolution) images and the corresponding target images: structural similarity index (SSIM) [27], and peak signal to noise ratio (PSNR). The SSIM measure is based on computing the mean, variance and covariance of a variety of windows in the compared images. The PSNR is based on the inverse of the mean squared error (MSE). Generally, for both measures, higher values indicate the generated image is more similar to the target image. During training steps we recorded the PSNR in each epoch to evaluate the convergence speed and saved the best PSNR performance models. Figure 3 shows generated images with WGAN and WGAN-GP trained on the CIFAR-10 dataset usidg a fixed training strategy and the proposed adaptive training strategy. As can be observed, the proposed adaptive WGAN training strategy progresses faster than a fixed training strategy. When comparing the best epoch (the epoch with the best FID) results, we observe that the adaptive WGAN results are more realistic compared with the original WGAN results. Both WGAN-GP and the adaptive WGAN-GP can get some meaningful results, but the adaptive strategy progresses faster. Figure 4 shows images generated with WGAN and the proposed adaptive WGAN trained on the LSUN conference rooms and the LFW datasets. We observe that the adaptive WGAN results have more details and are more realistic on both datasets.

Quantitative evaluation
To evaluate WGAN and WGAN-GP for image synthesis we use the Inception Score (IS) and the Frenchet Inception Distance (FID). We train the networks using both the fixed strategy and the proposed adaptive strategy (with different importance parameter λ values) on the CIFAR-10 dataset and record the first epoch when a target FID value is obtained. We record in addition the total number of G and D updates. The results are shown in Table 1. As can be observed the proposed adaptive training scheme converges faster and to a better result when λ is between 1 and 5 (with best result at λ = 3).
When considering the total number of G and D updates, we see that with λ = 3 the proposed adaptive training strategy trains G three to four times more than D, whereas the  Table 3. Comparison of various WGAN training schemes targeting balancing G and D training (see Section 2). Methods with "adaptive" in their name employ the proposed adaptive training scheme. The network was trained using the CIFAR-10 dataset (batch-size=64,imagesize=32x32, 1000 epochs). The symbol '/' indicates that no data is available from the original paper. As can be observed, the proposed adaptive training scheme converges faster.
suggested ratio in the fixed training scheme [8] [24] [3] is to train D five times more than G. The update frequency of G and D can be effected by multiple factors such as the complexity of the models, the optimizer and its parameters (e.g. learning rate), and the loss function. It is therefore difficult to empirically estimate the G and D training ratio. Moreover, even if the training ratio of G and D is somehow determined (e.g. hyper-parameter search) it may change during iterations as the algorithm gets close to convergence. The proposed adaptive training strategy alleviates the need to carefully set this parameter and provides a systematic way to continuously estimate it. While the proposed adaptive scheme still involves selecting an importance parameter λ, training results are less sensitive to the selection of this pa-rameter and a simple default (e.g. λ = 1) may suffice.
A similar evaluation of the proposed adaptive WGAN training strategy when trained with different datasets is provided in Table 2. In this table training is done separately both with the LSUN conference room and LFW datasets. The evaluation on these datasets results in similar conclusions to the ones obtained when training with the CIFAR-10 dataset. The proposed adaptive training strategy converges faster and to a better result.
Comparison of various WGAN training schemes targeting balancing G and D training (see Section 2) is provided in Table 3. The compared methods include: WGAN-GP [9], WGAN-GP with TTUR [11], Unbalanced GAN [10], and Gradient Penalty [19]. Training is done using the CIFAR- 10 dataset. Methods with "adaptive" in their name employ the proposed adaptive training scheme. As can be observed, the proposed adaptive training scheme converges faster. The deblur GAN and super resolution WGAN networks are evaluated in Tables 4 and 5 respectively. The Deblur GAN network is trained using the Caltech-UCSD Birds-200-2011 dataset whereas the SR-WGAN network is trained on the DIV2K dataset. Training in both cases is done both with a fixed update strategy and the proposed adaptive training strategy. Evaluation is done using SSIM and PSNR. In the evaluation we record the first epoch during training where a target PSNR value is achieved. In addition we record the total number of update steps for the generator and the discriminator. We observe that the proposed adaptive training scheme converges faster and to a better result for both the deblur and super resolution networks. Further, here too the ratio of training steps for G and D obtained by the proposed adaptive training strategy does not match the recommended ratio of training steps thus supporting the need for the proposed adaptive training scheme.

Conclusion
In this paper we propose an adaptive WGAN training strategy which automatically determines the sequence of generator and discriminator training steps. The proposed approach compares the loss change ratio of the generator and discriminator to decide on the next component (G or D) to be trained and so balances the training rate of the generator and discriminator. We show that a WGAN with this strategy could reach the local Nash Equilibrium point for the Dirac-GAN. Experimental evaluation results using different networks and datasets show that the proposed adaptive training scheme normally converges faster and to a lower minimum. Another advantage of the proposed adaptive update strategy is that it alleviates the need to empirically determine the number of update steps for the generator and discriminator. In future work, we will investigate additional update strategies suitable for various GAN structures and loss terms.