{'fraction_non_alphanumeric': 0.030621447597474778, 'fraction_numerical': 0.01363283625914973, 'mean_word_length': 4.862535349809418, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Double Descent in Adversarial Training: An Implicit Label Noise Perspective

Here, we show that the robust overfitting shall be viewed as the early part of an epoch-wise double descent — the robust test error will start to decrease again after training the model for a considerable number of epochs. Inspired by our observations, we further advance the analyses of double descent to understand robust overfitting better. In standard training, double descent has been shown to be a result of label flipping noise. However, this reasoning is not applicable in our setting, since adversarial perturbations are believed not to change the label. Going beyond label flipping noise, we propose to measure the mismatch between the assigned and (unknown) true label distributions, denoted as implicit label noise. We show that the traditional labeling of adversarial examples inherited from their clean counterparts will lead to implicit label noise. Towards better labeling, we show that predicted distribution from a classifier, after scaling and interpolation, can provably reduce the implicit label noise under mild assumptions. In light of our analyses, we tailored the training objective accordingly to effectively mitigate the double descent and verified its effectiveness on three benchmark datasets.


Introduction
In adversarial training, a typical phenomenon is that after a certain training epoch, the robust test error will start to increase constantly with further training, despite the robust training error continues to decrease (Rice et al., 2020). This phenomenon, known as robust overfitting, is believed to be separated from double descent (Belkin et al., 2019) in the literature (Rice et al., 2020).
Here, we find that robust overfitting shall be viewed as the early part of an epoch-wise double descent (Nakkiran et al., 2020). As shown in Figure 1, for relatively large models, the robust test error increases only transiently and will eventually decrease again after a considerable number of training epochs. Moreover, we observe consistent and similar patterns across various training settings (e.g., different sample sizes, optimizers, learning rate schedulers, and neural architectures), which further verifies our intuition on the connection between the robust overfitting and the double descent (more discussions are elaborated in Appendix B.1).
Inspired by this observation, we further analyze the epoch-wise double descent to better understand the robust overfitting. In standard training, the double descent has been shown to be related to the label flipping noise (Nakkiran et al., 2020;Yang et al., 2020b). However, adversarial perturbations are believed to not change underlying labels, which is further confirmed empirically from our manual inspection. Therefore, the popular label flipping noise explanation is not applicable here. On the other hand, it is reasonable to believe that adversarial perturbation distorts the label distribution since the examples do become more ambiguous after perturbation. Thus, we focus on the implicit label noise which measures the mismatch between the assigned label distribution and (unknown) true label distribution. Extensive analyses of such implicit label noise reveal the intriguing dependency of robust overfitting on perturbation size (Dong et al., 2021b) and data quality (Dong et al., 2021a). Guided by our analyses, we design a theoreticallygrounded method to mitigate the robust overfitting.
The key idea is to resort to an alternative labeling of the adversarial examples. We show that the predictive label distribution of a probabilistic classifier adversarially trained as usual, but after being properly scaled and interpolated, can be utilized as a better labeling of the adversarial examples and provably reduce the implicit label noise. This echoes the recent empirical practice of incorporating knowledge distillation (Hinton et al., 2015) into adversarial training (Chen et al., 2021). While previous works intuitively select fixed scaling and interpolation parameters for knowledge distillation, we show that it is possible to fully unleash the potential of knowledge distillation by automatically determining the set of parameters that maximally reduces the implicit label noise with a strategy similar to confidence calibration (Guo et al., 2017). Such strategy can further mitigate robust overfitting to a minimal amount without additional human tuning effort. Extensive experiments on different datasets, training methods, neural architectures and robustness evaluation metrics verify the effectiveness of our method.
In summary, our findings and contributions are as follows.
• We show that robust overfitting shall be viewed as the early part of an epoch-wise double descent, extending the common belief in adversarial training. The remainder of this paper is organized as follows. In Section 2, we briefly review the existing works that explore robust overfitting and double descent. In Section 3, we theoretically explore the origin of double descent in adversarial training from an implicit label noise perspective. In Section 4, we propose to mitigate the double descent in adversarial training by alternative labeling based on our understanding. Section 5 demonstrates the effectiveness of our method on realistic datasets. Conclusions and further implications are discussed in Section 6.

Related Work
Robust overfitting and double descent in adversarial training. Double descent refers to the phenomenon that overfitting by increasing model complexity will eventually improve test set performance (Neyshabur et al., 2017;Belkin et al., 2019). This appears to conflict with the robust overfitting phenomenon in adversarial training, where increasing model complexity by training longer will impair test set performance constantly after a certain point during training. It is thus believed in the literature that double descent and robust overfitting are separate phenomena (Rice et al., 2020).
Towards a more complete understanding of robust overfitting, in this work, we conduct adversarial training for exponentially more epochs than the typical practice. We find that robust overfitting shall be viewed as the early part of an epoch-wise double descent. And increasing the model architecture size, another way to increase the model complexity, can modulate the epoch-wise double descent curve such that either the overfitting curve is shown, or the entire double descent curve is revealed within the same number of training epochs as shown in Figure 1. Therefore, robust overfitting will not go beyond modern generalization theory as an exception and should be adequately explained by the origin of double descent such as label noise.
A recent work also considers a different notion of double descent that is defined with respect to the perturbation size (Yu et al., 2021). Such double descent might be more related to the robustnessaccuracy trade-off problem (Papernot et al., 2016;Su et al., 2018;Tsipras et al., 2019;Zhang et al., 2019), rather than the classic picture of double descent based on model complexity.
Understand double descent in adversarial training.
In standard training, double descent is often attributed to increased variance, with label noise being a common source. Definitions of label noise vary in literature. Theoretically-grounded analyses of double descent focus on additive label noise (Advani & Saxe, 2020;Mei & Montanari, 2019;Hastie et al., 2019;Belkin et al., 2020;d'Ascoli et al., 2020), but only applicable to regression problems. Theoretical results on double descent are scarce on classification problems, with a few works introducing noise by randomly masking the feature vector (Deng et al., 2019;Kini & Thrampoulidis, 2020). Analyses of double descent on classification problems are more common in empirical studies, where a typical way to induce double descent is to inject label flipping noise, namely the labels of a random fraction of training examples are flipped to other labels (Nakkiran et al., 2020;Yang et al., 2020b). However, such a definition of label noise cannot properly fit the scenario in adversarial training, where labels are not likely to be flipped due to small adversarial perturbation.
To understand the double descent in adversarial training within the existing picture of double descent, we introduce implicit label noise which is incurred by the mismatch between the assigned label distribution and true label distribution of the adversarial example. Such label noise can be interpreted as an instance-dependent, class dependent label noise in reference to the systematic studies on the taxonomy of label noise (Frénay & Verleysen, 2014;Song et al., 2020), thus can be the origin of double descent in adversarial training.
Mitigate robust overfitting. Robust overfitting hinders the practical deployment of adversarial training methods as the final performance is often sub-optimal. Various regularization methods including classic approaches such as 1 and 2 regularization and modern approaches such as cutout (Devries & Taylor, 2017) and mixup (Zhang et al., 2018) have been attempted to tackle robust overfitting, whereas they are shown to perform no better than simply early stopping the training on a validation set (Rice et al., 2020). However, early stopping raises additional concern as the best checkpoint of the robust test accuracy and that of the standard accuracy often do not coincide , thus inevitably sacrificing the performance on either criterion. Various regularization methods specifically designed for adversarial training are thus being proposed to outperform early stopping, including regularization the flatness of the weight loss landscape (Wu et al., 2020;Stutz et al., 2021), introducing low-curvature activation functions (Singla et al., 2021) and adopting stochastic weight averaging (Izmailov et al., 2018) and knowledge distillation (Hinton et al., 2015) (Chen et al., 2021). These methods are likely to suppress the implicit label noise, with the self-distillation framework (i.e. the teacher shares the same architecture as the student model) introduced by (Chen et al., 2021) as a particular example since introducing teacher's outputs as supervision is almost equivalent to the alternative labeling inspired by our understanding of the origin of label noise in adversarial training.

Double Descent from an Implicit Label Noise Perspective
In this section, we present a novel perspective to understand the double descent in adversarial training.

Preliminaries
Let X ⊂ R d define the input space equipped with a norm · : X → R + and Y = {1, . . . , c} define the label space. Considering a classification problem with a training set D = {(x, y)}, where x ∈ X is an input and y ∈ Y is its assigned label. We denote p(Y = j|x) as the distribution of the assigned labels of x given by a group of annotators. Typically y = arg max j p(Y = j|x). When there is no ambiguity, we will denote p(Y = j|x) as p(y|x) with a slight abuse of notation. Let y * ∈ Y be the true label of x and p(y * |x) be its distribution, which can be given by a group of experts. Note y * = arg max j p(Y * = j|x).
Assumption 3.1. We assume there is no label noise in D, namely y = y * and p(y|x) = p(y * |x).

Figure 2:
The illustration of the origin of implicit label noise in adversarial training. The traditional adversarial label introduces the implicit label noise by inducing a distribution mismatch between the assigned label distribution and true label distribution of the adversarial example. Our rectified model probability as an alternative labeling can provably reduce this distribution match.
Definition 3.1 (Data quality). Given an example x ∈ X , we define its data quality as max j p(Y * = j|x), namely the maximum class probability of the true label distribution. Assumption 3.2. We assume the original clean dataset D contains mostly high-quality data, namely max j p(Y * = j|x) ≈ 1.
We now briefly review adversarial training. Let f (·; θ) : X → Y be a probabilistic classifier and f (·; θ) j be its j-th class probability. Adversarial training can be viewed as an data augmentation technique that trains the classifier f on a set of adversarial examples D δ = {(x δ , y δ )}, namely (1) Here, x δ ≡ x + δ is the adversarial example of x produced by the inner maximization, where and y δ ∈ Y is the assigned label of x δ . Finally, we denote y * δ ∈ Y as the true label of x δ . Remark 3.1. In adversarial training, it is the common practice that copies the label of a clean example to its adversarial counterpart, namely y δ ← y and p(y δ |x δ ) ← p(y|x).
We denoteỹ δ ≡ y and its associated distribution as the traditional adversarial label. We believe traditional adversarial label is the key problem that induces double descent in adversarial training.

Traditional adversarial label does not introduce label flipping noise
Since label noise is often essential to explain double descent in standard learning for modern neural architectures (Nakkiran et al., 2020;Yang et al., 2020b), we wish to check if the traditional adversarial label produces any label flipping noise, a typical type of label noise, namely ifỹ δ = y * δ for any adversarial example x δ . This is equivalent to checking if y * = y * δ , sinceỹ δ = y by the construction of the traditional adversarial label (Remark 3.1) and y = y * by the construction of our clean dataset (Assumption 3.1). Note that y * = y * δ means the semantics of the adversarial example is distorted significantly such that its true label is now different from the true label of its clean counterpart.
We first visually check the adversarial examples created by the inner maximization in adversarial training. We are specifically interested in those examples with the lowest data quality as they are mostly likely to become label noise after perturbation. We estimate the data quality based on model ensemble as shown in Appendix E.1. One can find that the adversarial counterparts of those low-quality examples still match their original labels, albeit being slightly more ambiguous.
We have also found some evidence from the literature. Yang et al. (2020a) draw the same conclusion by studying the distance between examples in the input space. In the CIFAR-10 training set, they found that the minimum distance between any two examples from two different classes respectively is around 50/255 in terms of ∞ norm, while the perturbation radius typically used in adversarial training is around 8/255, which is significantly smaller. It is thus reasonable to claim that a significant amount of label flipping noise will not be produced by the inner maximization, and subsequently it cannot explain the double descent observed in adversarial training.

Traditional adversarial label introduces implicit label noise
Although traditional adversarial label is unlikely to directly produce label flipping noise, we show that it can produce label noise implicitly.
It has been widely observed that adversarial examples transfer between different classifiers even with distinct architectures (Papernot et al., 2017). Meanwhile, an adversarial example often contains salient characteristics of classes other than the original label from human perspective (Tsipras et al., 2019;Ilyas et al., 2019). Therefore, the true label distribution of an adversarial example is likely to be different from its clean counterpart, which means p(y * δ |x δ ) = p(y * |x). Note this will not conflict with y * δ = y * as discussed in Section 3.2, since the latter only implies arg max j p(Y * δ = j|x δ ) = arg max j p(Y * = j|x). Now we show the traditional adversarial label is improper in terms of the label distribution. As illustrated in Figure 2, we have p(ỹ δ |x δ ) = p(y|x) by Remark 3.1 and p(y|x) = p(y * |x) by Assumption 3.1, which together with the effect of adversarial perturbation p(y * δ |x δ ) = p(y * |x) imply that p(ỹ δ |x δ ) = p(y * δ |x δ ). This means there is a distribution mismatch between the true label distribution and the assigned label distribution of the adversarial example. Such distribution mismatch introduces implicit label noise in the training set D δ employed in adversarial training. Definition 3.2 (Implicit label noise). Given an example x ∈ X , we define the implicit label noise as the probability that its assigned label is different from its true label, namely p(Y = Y * |x). Remark 3.2. Implicit label noise is equivalent to instance-dependent and class-dependent label noise, since p Before showing the connection between the distribution mismatch and the implicit label noise, we need to quantify the distribution mismatch first. Here we adopt the total variation distance. Definition 3.3 (Total variation (TV) distance). For two discrete probability distributions p(y) and p(y ) where y, y ∈ Y, the total variation distance between them can be defined as We are now ready to present our main result. Theorem 3.1 (Traditional adversarial label incurs implicit label noise). The implicit label noise incurred by the distribution mismatch between the traditional adversarial label and the true label of the adversarial example is lower-bounded, namely An informal proof can be sketched from a frequentist's view and help the understanding of implicit label noise. Say there are M identical copies of x δ in the training set D δ , with their true labels and traditional adversarial labels distributing according to p(y * δ |x δ ) and p(ỹ δ |x δ ), respectively. The number of copies that have the same true label and assigned label is TV by the definition of the total variation distance.

Dependence of implicit label noise in adversarial training
We now show the implicit label noise in adversarial training depends on the perturbation size and the data quality, which will subsequently affect the double descent curves.
We consider optimal adversarial perturbation for simplicity, which is defined as where f * (·) denotes the optimal probabilistic classifier such that f * (x) = p(y * |x). With being the cross-entropy loss, the above equation can be rewritten as δ * = arg min which implies the optimal perturbation will reduce the probability mass of the true label. With the optimal adversarial perturbation, the minimum implicit label noise can be solved as follows. Theorem 3.2 (Minimum implicit label noise under optimal perturbation). Assume the true label distribution p(y * |x) is locally convex around x and can be asymptotically described as where min means the lower bound of the minimum label noise.
In Appendix A.1, we show that Assumption (7) holds true for a Gaussian mixture model. The above theorem shows that, when the data quality of the clean example is relatively high, the implicit label noise introduced by adversarial perturbation is proportional to (1) the perturbation radius (2) the data quality. We conduct controlled experiments and empirically verify this correlation in Appendix B.2.
Interestingly, the above analysis implies that if the perturbation does not reduce the probability mass of the true label, it will not cause implicit label noise even with an extremely large perturbation radius. We demonstrate this in detail and empirically verified it for Gaussian noise in Appendix B.3.  (Jacot et al., 2020;Yang et al., 2020b;d'Ascoli et al., 2020). Since implicit label noise is just a special case of label noise (Remark 3.2), and adversarial training can be viewed as standard training on an augmented dataset (Equation (1)), it can be inferred that implicit label noise will cause double descent in adversarial training. To demonstrate this in a straightforward way, in Figure 3 we employ standard training on a dataset augmented by fixed adversarial perturbation and show it can indeed produce double descent.

Implicit label noise induces double descent
Since implicit label noise modulates the double descent, and by Theorem 3.2 it depends on the perturbation radius and data quality, the double descent in adversarial training should strongly correlate with the perturbation radius and data quality. Indeed, it has been observed respectively that small perturbation radius will not induce robust overfitting (Dong et al., 2021b), and high-quality data will not induce robust overfitting (Dong et al., 2021a).

Mitigate Double Descent in Adversarial Training
In this section, we focus on suppressing the implicit label noise in adversarial training to mitigate double descent from both theoretical and practical perspectives. Since the implicit label noise is incurred by the traditional adversarial label which improperly labels the adversarial example and causes a distribution mismatch between its assigned label and true label, we wish to find an alternative label (distribution) for the adversarial example to reduce such distribution mismatch.

Rectify model probability to reduce distribution mismatch
We show that it is possible to reduce the distribution mismatch by utilizing the predictive probability of a classifier trained on traditional adversarial labels, which we will refer as the model probability for simplicity. We now provide a theoretical guarantee to show that, with temperature scaling (Hinton et al., 2015;Guo et al., 2017) enabled in the softmax function, model probability induces a distribution mismatch provably smaller than the traditional adversarial label. Theorem 4.1 (Model probability induces smaller distribution mismatch than the traditional adversarial label). Let f (x; θ, T ) denote the predictive probability of a hypothesis classifier scaled by temperature T , namely where z is the logits of the classifier from x.
Let x δ be an adversarial example correctly classified by a classifier f , i.e. arg max j f (x δ ; θ) j = y * δ , then there exists T , such that One possible way to further reduce the distribution mismatch is to interpolate between the model probability and the traditional adversarial label. Theorem 4.2 (Interpolation can further reduce the distribution mismatch). Let x δ be an adversarial example incorrectly classified by a classifier f , i.e. arg max Note the above theorem focus on incorrectly classified examples and thus can be regarded as a complement to Theorem 4.1.
As a summarization, to reduce the distribution mismatch, we propose to use the following distribution as the assigned label of the adversarial example in adversarial training instead of the traditional adversarial label.
We refer this label distribution as the Rectified model probability.
In Appendix D, we show that the optimal hyper-parameters (i.e. T and λ) of almost all training examples concentrate on the same set of values by empirically studying on a synthetic dataset with known true label distribution. Therefore it is possible to find an universal set of hyper-parameters that reduce the distribution mismatch for all adversarial examples.

Determine the optimal temperature and interpolation ratio
The set of temperature and interpolation ratio in the rectified model probability that maximally reduces the distribution mismatch is not straightforward to find as the true label distribution of the adversarial example is unknown in reality. Fortunately, given a sufficiently large validation dataset as a whole, it is possible to measure the overall distribution mismatch in a frequentist's view without knowing the true label distribution of every single example. A popular metric adopted here is the negative log-likelihood (NLL) loss, which is known as a proper scoring rule (Gneiting & Raftery, 2007) and is also employed in the confidence calibration of deep networks (Guo et al., 2017). By Gibbs's inequality it is easy to show that the NLL loss will only be minimized when the assigned label distribution matches the true label distribution (Hastie et al., 2001), namely And here y * δ is known since the adversarial example should share the same argmax label with its clean counterpart as shown in Section 3.2. Therefore, we propose to find the optimal T and λ in Equation (8) as One may note that Equation (10) cannot be directly optimized since the traditional adversarial label is only defined on the example in the training set and cannot be simply generalized to the validation set. A reasonable solution is using the nearest neighbour classifier to find the closest traditional adversarial label for every example in the validation set. However, to speed up the optimization we propose to employ the classifier overfitted by the traditional adversarial labels on the training set as an surrogate (see Appendix E.2), which works well in practice. Such process incurs almost no additional computation as we simply obtain the logits of a surrogate classifier on the validation set.

Rectified model probability mitigates double descent
We now work on a realistic dataset (CIFAR-10) to demonstrate the rectified model probability proposed in Equation (8) can effectively mitigate the robust overfitting, or equivalently the epoch-wise double descent in adversarial training. The outer minimization of adversarial training (Equation (1)) now becomes where θ Trad denotes the parameters of a classifier trained on the traditional adversarial label beforehand. The details of the experimental setting are available in Appendix F.1.
As shown in Figure 4, adversarial training on rectified model probability can best mitigate the robust overfitting when the temperature T and interpolation ratio λ are optimal. Such optimal hyperparameters perfectly aligns with the ones automatically determined by Equation (10).

Results & Discussions.
We compare our method (Ours) with two baselines: regular PGD training on traditional adversarial label (AT), and PGD training combined with self-distillation (KD-AT) with fixed temperature T = 2 and interpolation ratio λ = 0.5 as suggested by Chen et al. (2021). As shown in Figure 5, our method can effectively mitigate robust overfitting for all datasets, with both standard accuracy (SA) and robust accuracy (RA) constantly increasing throughout training.
In Table 1, we measure the difference between the RA at the best checkpoint (Best) and at the last checkpoint (Last) to clearly show the overfitting gap. Our method can reduce the overfitting gap to less than 0.3% for all datasets. One may note that self-distillation with fixed hyper-parameters is in fact inferior in terms of reducing robust overfitting, while its effectiveness can be significantly improved with the optimal hyper-parameters automatically determined by our method, which further verifies our understanding of robust overfitting.
Compared with self-distillation with fixed hyper-parameters, our method can also boost both RA and SA at the best checkpoint for all datasets.

Conclusion and Discussions
In this paper, we extend the understanding of robust overfitting and show that it is the early part of an epoch-wise double descent in adversarial training. Our further analyses show that the double descent may originate from the implicit label noise introduced by the improper labeling of the adversarial examples. Based on our understanding, we propose an alternative labeling of adversarial examples by rectifying model probability. Our method can effectively mitigate robust overfitting without any manual hyper-parameter tuning. In the future, we also plan to explore if it is possible to rectify the model probability on the fly during the adversarial training for a better efficiency.
By showing robust overfitting and epoch-wise double descent are in fact the same phenomena, our work eases the difficulty in understanding overfitting in modern generalization theory. Existing theories on double descent can be readily applied to adversarially robust learning setting, and potentially consolidate the theoretical grounding of robust learning.
Beyond the double descent in adversarial training, our definition of implicit label noise from a distribution mismatch perspective may also exist in a variety of scenarios, especially in real-world tasks where labeling is often complicated and ambiguous. Our method to rectify such label noise should thus be applicable as well. This would be an interesting future work.
Finally, on the practical side, our understanding of robust overfitting opens new opportunities to advance adversarial training practice. Better labeling of the adversarial examples may be still in demand to further reduce robust overfitting.

Reproduciblity Statement
We conduct experiments on public benchmark. We will release implementations for all methods and scripts for all experiments on GitHub, under the Apache-2.0 license.

Ethic Statement
In this paper, we explore the double descent phenomenon in adversarial robust learning from the implicit label noise perspective. We experiment on three benchmark datasets that are publicly available. Our analyses offer more in-depth understandings about adversarial training and can better improve the robustness of neural network models. Therefore, we believe our work is ethically on the right side of spectrum and has no potential for misuse, and cannot harm any vulnerable population.

A Proofs
A.1 Proofs and remarks for implicit label noise Total variation distance.
For two discrete probability distributions p(y) and p(y ) where y, y ∈ Y, the total variation distance between them can be equally defined as Proof of Theorem 3.1. This theorem can be easily proven by the coupling inequality.

Proof of Theorem 3.2.
Proof. For simplicity, we consider the adversarial perturbation generated by FGSM. Other adversarial perturbation can be viewed as a Taylor series of such perturbation.
With the assumption we then have min p(ŷ = y|x + δ) ∝ ε(1 − p(y = j * |x)), where min means the lower bound of the minimum label noise.

Sanity check of Assumption 7.
Here we check if Assumption 7 is reasonable by studying a Gaussian mixture model (GMM).
The conditional distribution in a Gaussian mixture model can be formulated as The gradient can be derived as

A.2 Proofs for mitigating double descent
Proof of Theorem 4.1.
where the inequality holds by the triangle inequality.
Meanwhile, we have Therefore, it can seen that for T = T * , Proof of Theorem 4.2.

Now we prove Theorem 4.2
Proof. First let p(y δ |x δ ) = p(y|x) ≈ 1(y). This is our assumption. But the approx here would be a problem, we need exactly one-hot.

B More empirical analyses B.1 Epoch-wise double descent is ubiquitous in adversarial training
In this section, we conduct extensive experiments with different optimizers, sample sizes, model architectures, and learning rate schedulers to verify the connection between robust overfitting and epoch-wise double descent. The default experiment settings are listed in Appendix F.2 in detail.
Optimizer. Similar to the setting employed in Nakkiran et al. (2020), we conduct the adversarial training using both the Adam optimizer and SGD. As already shown in Figure 1, double descent can be observed for both optimizers, although Adam may be inferior compared to SGD.
Sample size. We randomly sample a desired number of examples without replacement from the original training set in CIFAR-10. As shown in Figure 6, for both optimizers, increasing sample size will shrink the area under the double descent curve, but will not significantly distort its shape. Similar observation is also made for double descent curve under standard training (Nakkiran et al., 2020). Sample Size (×10 3 ) Figure 6: Varying sample size will shrink the area under the epoch-wise double descent curve, but will not significantly distort its shape.
Model capacity. We modulate the capacity of the deep model by varying the widening factor of the Wide ResNet. To extend the lower limit of the capacity, we allow the widening factor to be less than 1, in which case the number of channels in each residual block is scaled similarly but rounded. The number of channels in the first convolutional layer will be reduced accordingly to ensure the width monotonically increasing through the forward propagation. To accelerate the training with an extremely large model, we randomly sample a training set of size 5000 and employ the Adam optimizer, since the sample size will not significantly distort the shape of the double descent as shown above. Figure 7a shows that the double descent will gradually become more complete as the model capacity increases and the model translates from under-parameterized to over-parameterized regime (Nakkiran et al., 2020).  Robust Test Error (%) Model architecture  (b) Different model architectures will affect the double descent curve. In particular, VGG-11 will have the second descent delayed due to its inferior performance compared to residual architectures. . We select these configurations to ensure approximately comparable model capacities 1 . As shown in Figure  7, different model architectures may produce slightly different double descent curves. The second descent of VGG-11 in particular will be delayed due to its inferior performance compared to residual architectures.

Learning rate scheduler.
A specific learning rate scheduler may shape the robust overfitting differently as suggested by Rice et al. (2020). We consider the following learning rate schedulers in our experiments.
• Piecewise decay: The initial learning rate rate is set as 0.1 and is decayed by a factor of 10 at the 100th and 500th epochs within a total of 1000 epochs. • Cyclic: This scheduler was initially proposed by Smith (2017) and has been popular in adversarial training. We set the maximum learning rate to be 0.2, and the learning rate will linearly increase from 0 to 0.2 for the initial 400 epochs and decrease to 0 for the later 600 epochs. • Cosine: This scheduler was initially proposed by Loshchilov & Hutter (2017). The learning rate starts at 0.1 and gradually decrease to 0 following a cosine function for a total of 1000 epochs.
Experiments on various learning rate schedulers show the second descent can be widely observed except the piecewise decay, where the appearance of second descent might be delayed due to extremely small learning rate in the late stage of training. This further demonstrates the connection between robust overfitting and epoch-wise double descent.

B.2 Dependence of double descent in adversarial training
In this section, we show that the double descent in adversarial training is conditioned on the data quality. As models are trained on adversarial examples, data properties in adversarial training are concerned with the quality of the clean examples, and the adversary employed to generate the perturbation, which further depends on the perturbation radius and the number of attack iterations 2 . We show that the double descent in adversarial training has a strong dependence on the data quality and the perturbation radius, but almost no dependence on the number of attack iterations.
Perturbation radius. Overfitting has been shown to dominate in adversarial training, but rarely appear in standard training (Rice et al., 2020). This suggests the overfitting, or more generally double descent, is conditioned on the perturbation radius in adversarial training, since standard training and standard accuracy can be viewed as adversarial training and robust accuracy with zero perturbation radius, respectively. By modulating the perturbation radius, we show that such correlation is gradual. As shown in Figure 9, the double descent emerges and exacerbates as the perturbation radius increases, indicating a strong correlation between perturbation radius and double descent in adversarial training.
In particular, when the perturbation radius is around 4/255, a second minimum can be observed which is equivalently good or even better than the first minimum. This again validates the strong connection between robust overfitting and double descent, and suggests longer training can still be helpful for adversarial training.
Data quality. Previous works have shown that some datasets such as ImageNet (Deng et al., 2009) may produce significantly stronger robust overfitting (Rice et al., 2020). It has also been observed that the low-quality data in the same dataset causes the robust overfitting (Dong et al., 2021a). This implies that the double descent in adversarial training hinges on the quality of the data. We measure the data quality using the predictive probability of a classifier (see Appendix B.2 for details), and sample training sets with different levels of data quality controlled by a threshold of the quality-based rank. Figure 9 shows that as the quality of the training set degrades, the double descent gradually emerges and exacerbates, indicating a strong correlation between the data quality and the double descent in adversarial training. One may again note that when the quality of the training data is relatively high, a second minimum can be observed which is equivalently good as the first minimum.
The number of attack iterations. We have shown that the double descent in adversarial training strongly depends on the perturbation radius. In this section we conduct experiments to show whether it also depends on the strength of the adversary.
In Figure 10, we fix the perturbation radius as 4/255 where the double descent is relatively complete and vary the number of attack iterations of the PGD attack employed in the inner maximization. One may find that as long as the model capacity is reasonably large, the number of attack iterations will not significantly affect the double descent curve, both for epoch-wise one and model-wise one. From the analysis of implicit label noise, this is easy to understand as more attack iterations will not reduce the probability corresponding to the true label much more-it is widely observed more iterations in PGD attack only marginally increase the attack successful rate. Consequently, the distribution mismatch between the true label distribution and the assigned label distribution that induces the implicit label noise will not expand significantly. For the model-wise double descent, only the test error at the best checkpoint is shown to avoid overlapped curves since the last checkpoint achieves similar error as the best checkpoint in this training setting.

B.3 Gaussian noise
One may expect adversarial training and adversarial augmentation (see Section 3.5) produce double descent simply by perturbing the inputs, thus increasing the variance. To verify the implicit label noise induced by a mismatch between the true label distribution and the assigned label distribution is essential to produce double descent, we select one type of common corruption, Gaussian noise, to perturb the image inputs. We first briefly show that Gaussian noise does not cause a distribution mismatch.
We follow the notation introduced in Section 3.1, where δ ∈ R d now refers to a Gaussian perturbation.
Under a 1-st order approximation of p(y|x), the mismatch between the true label distribution and label distribution can be derived as (see proof of Theorem 3.2 in Appendix A.1) where δ ∼ N (0, σ · I d ). Since the image inputs span a low-dimension subspace X and the dimensionality d is large (3072 for CIFAR-10), it is highly likely that δ⊥X , which means ∇ x f * (x) j · δ = 0 and δ T ∇ 2 f * (z) j δ = 0 for all j. One can also empirically verify that a Gaussian perturbation is almost always orthogonal to the difference between any two images, while an adversarial perturbation is not.
We now experiment on dataset perturbed by Gaussian noise and verify our intuition. Similar to adversarial augmentation, we apply Gaussian noise to the training set only once, and then conduct standard training on the perturbed training set. Aligned with the setting of adversarial augmentation (See Appendix F.3), we employ Adam to train a WRN-28-5 on randomly selected 5000 examples for 1000 epochs. As shown in Figure 11, Gaussian noise with a perturbation radius as high as 80/255, which will reduce the accuracy of a classifier to the same level as an adversarial attack will, does not produce significant double descent. We note that similar observation has been made on common curruption benchmark CIFAR-10-C (Hendrycks & Dietterich, 2019) in a previous work (Yang et al., 2020b).

C More experiment results
In this section we conduct extensive experiments with different adversarial training methods, neural architectures and robustness evaluation metrics to verify the effectiveness of our method.     Synthetic Dataset. Since the true label distribution is typically unknown for adversarial examples in real-world datasets, we simulate the mechanism of implicit label noise in adversarial training from a feature learning perspective. Specifically, we adapt mixup (Zhang et al., 2018) for data augmentation on CIFAR-10. For every example x in the training set, we randomly select another example x in a different class and linearly interpolate them by a ratio ρ, namely x := ρx + (1 − ρ)x , which essentially perturbs x with features from other classes. Therefore, the true label distribution is arguably y ∼ ρ · 1(y)+(1−ρ)·1(y ). Unlike mixup, we intentionally set the assigned label asŷ ∼ 1(y), thus deliberately create a mismatch between the true label distribution and the assigned label distribution. We refer this strategy as mixup augmentation and only perform it once before the training. In this way, the true label distribution of every example in the synthetic dataset is fixed.

Concentration of optimal temperature and interpolation ratio of individual examples.
In Section 4.1 we have shown that in terms of individual examples, the rectified model probability can provably reduce the distribution mismatch between the assigned label distribution and true label distribution of the adversarial example. However, since the true label distribution is unknown in realistic scenarios, it is not possible to directly follow Theorems 4.1 and 4.2 and calculate the optimal set of hyper-parameters for each example in the training set. The best we can do is to employ a validation set and determine a universal set of hyper-parameters based on the NLL loss, which expects all training examples to share similar optimal temperatures and interpolation ratios. Here, based on the synthetic dataset where a true label distribution is known, we empirically verify this assumption is reasonable. In Figure 13 left, we solve the optimal temperature for each correctly classified training example based on Theorem 4.1 with the interpolation ratio fixed as 1.0. One can find that the individual optimal temperatures mostly concentrate between 0.5 and 1.5. In Figure 13 right, we solve the optimal interpolation ratio for each incorrectly classified training example based on Theorem 4.2 with the temperature fixed as 1.0 . One can find that the individual optimal interpolation ratio mostly concentrate between 0.5 and 0.7.

E Toolkit E.1 Estimation of the data quality
We use the predicative probabilities of classifiers trained on CIFAR-10 to score its training data. Similar strategy is employed in previous works to select high-quality unlabeled data to improve adversarial robustness (Uesato et al., 2019;Carmon et al., 2019;Gowal et al., 2020). Slightly deviating from these works focusing on out-of-distribution data, we use adversarially trained instead of regularly trained models to measure the quality of in-distribution data, since under standard training almost all training examples will be overfitted and gain overwhelmingly high confidence. Specifically, we adversarially train a pre-activation ResNet-18 with PGD and select the model at the best checkpoint in terms of the robustness. The quality of an example is estimated by the model probability corresponding to the true label without adversarial perturbation and random data augmentation (flipping and clipping). We repeat this process 10 times with random initialization to obtain a relatively accurate estimation.

E.2 Determine the optimal hyper-parameters
We employ a model overfitted on the training set to generate approximate traditional adversarial label of the adversarial example in the validation set. Such overfitted model is typically the model at the final checkpoint when conducting regular adversarial training for sufficient epochs. Mathematically, our final method to determine the optimal temperature and interpolation ratio in rectified model probability can be described as where f (x δ ; θ s , T ) denotes the predictive probability of a surrogate model scaled by temperature on x δ . F Experimental details F.1 Practical settings for adversarial training As a practical setting for adversarial training, We conduct the adversarial training on the entire training set for 160 epochs, with the learning rate starting at 0.1 and reduced by a factor of 10 at the 80 and 120 epochs. We employ SGD as the optimizer as the momentum and weight decay are set to 0.9 and 0.0005 respectively. In the inner maximization, we set the perturbation radius to be 8/255, the step size to be 2/255 and the number of attack iterations to be 10 for PGD and TRADES.

F.2 Settings for double descent in adversarial training
Training setting. We conduct the adversarial training for 1000 epochs unless otherwise noted. By default we use the Adam optimizer with the learning rate fixed as 0.0001, since it requires minimal hyper-parameter tuning. For SGD the learning rate starts at 0.1, and will not be changed unless otherwise noted. The batch size will be fixed to 128, and the momentum will be set as 0.9 wherever necessary. No regularization such as weight decay is used. These settings are mostly aligned with the empirical analyse of double descent under standard training (Nakkiran et al., 2020).

Sample size.
To reduce the computation load demanded by exponential training epochs In individual cases, we reduce the size of the training set by randomly sampled a subset of size 5000 from the original training set without replacement. This will linearly shift the double descent curve but will not significant distort its shape as shown in Appendix B.1. We adopt this setting for extensive experiments such as the analyses of the dependence of double descent on perturbation radius, data quality and the number of attack iterations. Note that in the experiment associated with data quality, we randomly sampled the training subset from those examples with quality lower than a threshold. The sampled subset is restricted to class-balanced.
Adversary setting. We conduct adversarial training with ∞ norm-bounded perturbations. We employ standard PGD training with the perturbation radius set to 8/255 unless otherwise noted. The number of attack iterations is fixed as 10, and the perturbation step size is fixed as 2/255.
Robustness evaluation. We consider the robustness against ∞ norm-bounded adversarial attack with perturbation radius 8/255. We use PGD attack with 10 attack iterations and step size set to 2/255.

Dataset.
We conduct experiments on the CIFAR-10 dataset, without additional data.
Hardware. We conduct experiments on NVIDIA Quadro RTX A6000.

F.3 Settings for adversarial augmentation
We generate adversarial examples with PGD attack on the model obtained at the best checkpoint through a pratical adversarial training (see Appendix F.1 for details). The number of attack iterations is fixed as 10 and the step size fixed as 2/255. The adversarial examples of all training examples along with their labels are then grouped into a new training set, where we conduct standard training for 1000 epochs. Other settings are same as those listed in Appendix F.2 except no adversary is employed.