{'fraction_non_alphanumeric': 0.03533405338682302, 'fraction_numerical': 0.010916525227588335, 'mean_word_length': 4.600561676387989, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Towards the Desirable Decision Boundary by Moderate-Margin Adversarial Training

Adversarial training, as one of the most effective defense methods against adversarial attacks, tends to learn an inclusive decision boundary to increase the robustness of deep learning models. However, due to the large and unnecessary increase in the margin along adversarial directions, adversarial training causes heavy cross-over between natural examples and adversarial examples, which is not conducive to balancing the trade-off between robustness and natural accuracy. In this paper, we propose a novel adversarial training scheme to achieve a better trade-off between robustness and natural accuracy. It aims to learn a moderate-inclusive decision boundary, which means that the margins of natural examples under the decision boundary are moderate. We call this scheme Moderate-Margin Adversarial Training (MMAT), which generates finer-grained adversarial examples to mitigate the cross-over problem. We also take advantage of logits from a teacher model that has been well-trained to guide the learning of our model. Finally, MMAT achieves high natural accuracy and robustness under both black-box and white-box attacks. On SVHN, for example, state-of-the-art robustness and natural accuracy are achieved.


Introduction
Deep neural networks (DNNs) have achieved great success in many tasks, including image classification; however, recent research indicates that DNNs are extremely vulnerable to adversarial attacks. The so-called adversarial attack is to mislead a classifier by adding some well-designed perturbations to a clean input. Although several countermeasures have been proposed [1,2,3], adversarial training has confirmed to be one of the most effective methods. Nevertheless, the higher Robust Accuracy (RA) of adversarial training is often accompanied by more Natural Accuracy (NA) degradation, which prompts us to achieve a good trade-off between RA and NA.
In fact, although adversarially trained models have learned inclusive decision boundaries that keep adversarial counterparts within the original class, the unwarranted increase in the margin along certain adversarial directions is harmful, as shown in Fig. 1(c). The decision boundary's over-inclusiveness causes a heavy cross-over between natural and adversarial examples, which is not conducive to balancing the RA-NA trade-off. In this paper, we expect to alleviate the excessive adversarial directional margin. We begin our analysis by examining the decision boundaries of the DNNs obtained by different training methods. Next, we investigate the details of previous work [4,5,6] and discover that it mostly treats all examples equally and assigns uniform parameters to them. We then conduct experiments to confirm that this is one of the main causes of excessive directional margin. Thus, we propose a novel adversarial training scheme, namely Moderate-Margin Adversarial Training (MMAT).
Here we separate the training examples clearly, identifying both the critical difference between correctly classified and misclassified examples by a model as well as the characteristics of each correctly classified example. For misclassified examples, it does not make sense to study their robustness. Therefore, we consider a boosted loss function to better learn the examples themselves; for correctly classified examples, we design a new attack strategy based on the variability of their margins. It can significantly reduce the excessive adversarial directional margin for the examples near the decision boundary. In addition, we obtain a well-trained model for the same architecture and use it to guide the training process of our defense model, which can regulate the desirable course of the decision boundary and, to some extent, weaken the over-adjustment of the decision boundary along adversarial directions. Ultimately, we obtain a moderate-inclusive decision boundary shown in Fig.  1 [4] proposed Projected Gradient Descent (PGD) method as a universal "first-order adversary", i.e., the most active attack utilizing the local first-order information about the network. Moreover, boundary-based methods like DeepFool [9] and optimization-based methods like CW [10] were also developed, making adversarial defense more challenging. Recently, the ensemble of diverse attack methods -auto-attack [11] by Croce et al., consisting of APGD-CE [11], APGD-DLR [11], FAB [12], and Square Attack [13], became a popular benchmark for testing the robustness of models. These are white-box attacks since the adversary has full knowledge of DNNs. Numerous works have also been published that investigate the transferability of adversarial examples for black-box attacks. Liu et al. [14] were the first to study the transferability of targeted adversarial examples. Dong et al. [15] showed that iterative attack methods incorporating the momentum term achieved better transferability. Xie et al. [16] boosted the transferability of adversarial examples by creating diverse input patterns with random resizing and random padding.
Adversarial training. Many of the early adversarial defenses [17,18,19] are broken by stronger attacks. Among various defense strategies, adversarial training is currently the best defense method against adversarial attacks. The initial idea of adversarial training is first brought to light by [7] [21] proposed searching for the least adversarial data for AT, which could be called FAT. Cui et al. [22] proposed LBGAT to adapt the logits of one model trained on clean data to guide adversarial training. Rade et al. [6] incorporated additional wrongly labelled examples during training and presented HAT to provide a notable improvement in accuracy without compromising robustness.

Preliminaries
is a dataset where x i ∈ R d is a natural example, and y i ∈ 1, 2, · · · , K is its corresponding label. A DNN f θ is a function with parameter θ to predict an input example is the logits output of the network with respect to class k, and p k (x i ) is the probability (softmax on logits) belonging to class k. An adversarial example x i is crafted by adding a small perturbation δ i to x i , which misleads DNNs as f θ (x i ) y i , while f θ (x i ) = y i . Small perturbation means that · is a generic distance norm that can be · ∞ and · 2 .
Decision boundary. Given a classifier f θ , a dataset D = (x i , y i ) N i=1 , we can define the decision boundary D ⊆ R d of f θ as the set: Margin. Given a unit vector v ∈ S d−1 , the margin m at x i along the direction v is given by: where v i is abtained by solving: where L(·) is an arbitrary loss function. This optimization problem is usually solved via PGD.

Motivation
A naturally trained model can attain perfect NA, but its average margin has a clear downward trend over the training process. Eventually, a large portion of the training examples as well as test examples lie close to the decision boundary after training [23]. The relatively low average margins lead to the vulnerability of DNNs to adversarial examples. In the adversarial setting, even though a slight downward trend is inevitable, it is not as severe as that in the natural setting, and this downward trend starts in later epochs compared to natural training. The adversarially trained models with larger average margins are robust while triggering a superfluous increase in the margin along adversarial directions as compared to the nominal increase required to attain robustness [6]. The excessive directional margin might be the reason for a reduction in natural accuracy. This argument is in line with the previous works by [24], [25].
We then obtain several robust models using the standard adversarial training (SAT) method [4], and further investigate the differences in RA and NA among them. As shown in Fig. 2, the RA grows as increases, but the gain of this growth becomes gradually smaller. Meanwhile, the process is accompanied by a continued drop in NA. We realize that one reason for this phenomenon is that all examples share a uniform . With the increase of , more examples have excessive directional margin, especially those near the decision boundary, which significantly cross over the decision boundary and are located in the area of natural examples, causing a serious cross-over mixture problem [21]. Therefore, we prefer designing finer-grained parameters i for different examples to alleviate the problem and exploit the potential of larger perturbation budgets.
Moreover, we can stimulate a slight push to the decision boundary during adversarial training to reduce the potential excess directional margin. This can be achieved by fine-tuning the robust model using the decision boundary learned by a natural model. To implement the idea, we consider a weak robust model as our teacher model, such as SAT-6 in Fig. 2, which inherits the decision boundary of the natural model better in the process of improving robustness.

Moderate-Margin Adversarial Training
To learn a moderate-inclusive decision boundary, we propose a novel adversarial training method named MMAT. This method achieves a good trade-off by designing two loss terms L 1 and L 2 respectively for adversarial and natural examples, and design a new attack strategy to generate our finer-grained adversarial examples.
calculating the margin of all examples: 14: else if m(x i ) ≤ M P 100 then 15:

Loss function
The first term of loss function, L 1 , is for the adversarial examples. Obviously, adversarial examples' classification is more arduous than natural examples', the decision boundary is no longer flat, and its local curvature profoundly affects the final performance of the defense model, as shown in Fig. 1, so the boosted cross entropy loss (BCE) is chosen here instead of the commonly used cross entropy loss. Thus L 1 is defined as follows: where p k (x i ) is the probability belonging to class k, the first term − log(p y i (x i )) is the commonly used CE loss, denoted CE ( p(x i ), y i ), and the second term − log(1 − max k y i p k (x i )) is a margin term used to improve the classifier's decision margin, allowing it to be a robust classifier with high confidence. The second term, L 2 , is for natural examples. The teacher model f t is a well-trained weak robust model (like SAT with = 6/255). The decision boundary of f θ will be fine-tuned in Parameter: Batch-size m, learning rate η, scaling parameter λ, attack step size α and number of attack iterations T 1: Train a teacher network f t on D 2: repeat 3: Read a mini-batch {(x i j , y i j )} m j=1 from D 4: for j = 1, 2, · · · , m do 5: where N(0, I) is the Gaussian distribution with zero mean and identity variance 6: Grade x i j based on its margin and set i j for it 7: for t = 1, 2, · · · , T do 8: 12: until training completed real time during training, and the fine-tuning will prevent the undesirable excessive rise in the margin to some extent, thus making it possible to achieve a better trade-off. We make use of logits from the teacher model to help reshape the decision boundary of our robust model, which reduces damage to the NA. Here we use MSE loss as L 2 : When the student f θ is trained with MSE loss, its representations attempt to follow the shape of the teacher's representations. Besides, MSE is efficient in transferring the teacher's information to a student [26].

Finer-grained adversarial examples
In this subsection, we elaborate on the generation of our finer-grained adversarial examples. Intuitively, we can learn worse examples with larger perturbation budgets than c to defend against attacks with that c . However, previous adversarial training mostly treats all examples equally to generate adversarial examples, which are also called rough adversarial examples here. It results in: 1) increasing the perturbation budget is accompanied by a significant decrease in natural accuracy; 2) large perturbation budgets are extremely unreasonable for misclassified examples or even correctly classified examples near the decision boundary; and 3) excessive perturbation budgets are not necessarily beneficial for robustness improvement. [N], f θ new (x i ) = y i }. As shown in Fig. 3, there is a fundamental difference between correctly classified and misclassified examples of a model, with the latter being inherently difficult to learn and inappropriate for further adversarial attack, i.e., i = 0. Actually, the margin distribution of correctly classified examples in Fig. 3 also shows significant differences between these examples, so we calculate margins for the examples correctly classified by the model f θ new for each epoch, and thus determine their attack parameters. Apparently, our design is more reasonable, which takes into account the multiple characteristics of the examples. In addition, for each epoch, we select a new subset D + and calculate the corresponding margin based on the current network f θ new . Because both the subset and the margin are changing, this new attack scheme belongs to a dynamic adjustment and demonstrates great flexibility.
Measure of example's margin. A key point of our attack strategy is the calculation of margin. Since calculating the exact margin in the input space is intractable, we have to search for an approximate solution. To be more precise, we search for a small perturbation δ such that f θ (x i + δ) f θ (x i ). Ideally, we then have m(x i , v) ≈ δ ∞ , although x i + δ is not necessarily an element of D . We will obtain the margin estimate with the help of DeepFool [9], an iterative adversarial attack which stops as soon as a perturbation δ has been found with In every iteration step of DeepFool, it linearizes the decision boundary around an example to gradually push the example over the closest boundary for minimal perturbation. This perturbation δ i can be written as: with index:l =l(x i ) := arg min Let I denote the stopping index of this iterative scheme for x i , i.e., f θ (x I i ) f θ (x i ). The desired adversarial perturbation is then defined as δ = I−1 i=0 δ i . Note that we only use successful adversarial perturbations for the margin approximation. Thus, if the DeepFool attack is not able to find a small adversarial perturbation for a given image, we regard its semantic features as more obvious, and the vulnerability can be ignored without considering its margin. We have m = δ ∞ now.
Implementation of attack strategy. We will use a strategy model to develop our attack strategy, an example-dependent strategy to set the parameter i when performing an adversarial attack, as shown in the Algorithm 1. Since most previous adversarial training work has used SAT-8 ( = 8/255) as a baseline and followed its attack setup, here we use a well-trained SAT-8 as a strategy model and consider the attack scheme to be migratory and applicable to other defense methods. The margin distribution of correctly classified examples of this strategy model is shown in Fig. 3(c). In fact, it is feasible for us to rank the margin values of examples in ascending order and to rate them into three grades according to a certain proportion. The examples at each grade are recorded as D A , D B , D C . The example set closest to the decision boundary, D A , is the key to determine the degree of destruction of natural accuracy and is our key concern, accounting for 40%; D B , the example set second closest to the decision boundary, the average margin is slightly greater than 8/255; D C , the example far from the decision boundary, basically completely avoid the cross-over between the adversarial examples and the original natural examples, so D B and D C can well improve the robustness of the model, and the two account for 30% respectively. The three grades are practically meaningful and sufficiently representative of the properties of correctly classified examples. D A , in particular, aims to ensure natural accuracy at the expense of robustness; D B has the greatest trade-off advantage; and D C can further exploit the potential robustness without requiring an excessive range of budgets. As a result, we set A , B , and c to the values shown in the Algorithm 1. Finally, the FAEs are solved by: Based on the two proposed loss terms and the FAEs, we can state our final objective function for MMAT: where λ is a tunable scaling parameter that balances the two parts of the hybrid loss. The Algorithm 2 depicts our Moderate-Margin Adversarial Training. Our attack strategy is based on the margins of examples. The teacher model is used to guide the real-time decision boundary adjustment of our model. As a result, MMAT can also be framed as performing student-teacher learning, particularly self-distillation [27] to mimic certain properties of the model itself.
The trade-off parameter, β, is present in TRADES, MART, LB-GAT, and HAT, and a higher β indicates that the robust accuracy is given more weight.
White-box attacks. We evaluate robustness with four types of attacks, i.e., FGSM, PGD 20 , CW ∞ , and Auto-Attack (AA), on two datasets. All attacks are constrained by the same perturbation budget, i.e., = 8/255. Note that the CW ∞ attack denotes using CW-loss within the PGD framework here. The goal of Auto-Attack is to reliably evaluate model robustness with an ensemble of diverse strong attack methods. We use the opensource code from [29] to test our models. Besides, for PGD, the training attack is PGD 10 with random start and step size α = /4, while the test attack is PGD 20 with random start, perturbation budget = 8/255 and step size α = /10. Training details. To evaluate the performance of different models, we conduct extensive experiments on ResNet-18 [30] using CIFAR-10 [31] and PreActResNet-18 using SVHN [32]. For CIFAR-10, following [20], the batch size is 128 and the number of training epochs is 100. All models are trained using the SGD optimizer with Nesterov momentum 0.9 [33], weight   decay 0.0035 and an initial learning rate of 0.01, which is divided by 10 at the 75-th and 90-th epochs. Data augmentation is performed. When performing data augmentation, we randomly crop the image to 32 × 32 with 4 pixels of padding, then perform random horizontal flips. For SVHN, following [6], the batch size is still 128. We use the SGD optimizer with Nesterov momentum 0.9 and weight decay 0.0005. We further employ cyclic learning rates [34] with cosine annealing and a maximum learning rate of 0.05 for SVHN. Here we only run 15 epochs.

Understanding the proposed MMAT
In this section, we investigate MMAT from three different perspectives: 1) the perturbation budgets of FAEs; 2) the guiding role of teacher models; and 3) replacing components of the BCE term.
Perturbation budget of FAEs. Since adversarial training is time-consuming, taking 3-30 times longer to form a robust network than a non-robust equivalent [35], [36], we prefer to use a metric without extra computation to determine the grade of examples instead of the margin. As shown in Fig. 4(a), z max (the largest element of a network's logit output after forward-propagation) of examples meet this requirement. In Fig. 4(b), the distribution of z max implies the hierarchical nature of these examples; we simply use Z 1 = 2, Z 2 = 6 to divide these examples into three grades, i.e., And, according to Algorithm 1, A = 5/255, we have B = 10/255, A = 15/255, which means that i of each example x i can be set. Furthermore, we investigate the effects of D A under various Z 1 and A , with the results summarized in Table 1. Apparently, larger Z 1 or smaller A appears to favor natural accuracy over robustness. The results demonstrate the powerful regulation capability of MMAT in terms of the RA-NA trade-off. We also discovered that MMAT can enable larger perturbation budget i by alleviating the cross-over mixture problem [21].
Role of teacher models. We first investigate the parameter a (1/λ) in the MMAT objective function defined in (9) which controls the strength of the regularization. A larger a indicates that the MSE term plays a more significant role, and the teacher model has a more obvious influence on the course of the decision boundary. As illustrated in Fig. 5(a) and (b), as a increases, the RA in the training process tends to decrease, especially when a = 1, while NA tends to increase. For a = 1/4 and a = 1/8, the model has higher RA and NA, which confirms that the fine-tuning effect of the teacher model can maintain high NA without compromising the robustness of the student model. Next, fixing a = 1/4, we further explore the effect of different teacher models. As the teacher model becomes more robust with lower NA, the student model also shows this trend. Thus, we confirm that fine-tuning of a suitable teacher model facilitates the desirable course of decision boundary to achieve a good trade-off.
Replacing components of L 1 term. Recall that loss term L 1 is defined by BCE. As we show in Fig. 6(a) and (b), learning with CE instead of our BCE suffers from insufficient learning with lower robustness throughout the entire training process. Similarly, when replacing CE with KL in the inner maximization of the adversarial min-max framework, the robustness has been weak throughout the training process. Fig. 6(c) and (d) show the contribution of our BCE term with FAEs. Specifically, we generate adversarial examples with three strategies. Both the RA and NA of AE-1 are lower than our original MMAT, suggesting that learning misclassified examples is more necessary than learning their adversarial counterparts. Moreover, AE-2 leads to a robustness degradation, indicating that our FAE associated with characteristics of natural examples better exploits the model's robustness potential.

Comparisons with other defense methods
In this section, we assess MMAT and other defense methods mentioned in Section 5.1 against white-box and black-box attacks, as well as perform a deep analysis to confirm the effectiveness of our method.  Evaluation results under white-box attacks. Table 2 shows the evaluation results of defense models under white-box attacks on CIFAR-10 and SVHN. Our model achieves high RA and NA across multiple datasets and attack methods. On CIFAR-10, TRADES and MART have relatively low NA but perform well in terms of robustness. LBGAT and HAT have higher NA, but their robustness is even weaker than SAT. It is worth noting that our model's robustness exceeds that of MART, while our NA remains very high, second only to HAT. Because it is easier to classify than CIAFR-10, the NA is generally high, and the difference between them on SVHN is small.
Despite this, our method has the best NA. In terms of robustness, TRADES achieves the best results under CW ∞ and AA attacks, LBGAT comes in second only to TRADES, MART loses its original advantage, while HAT and MMAT perform admirably under all four attacks.
Evaluation results under black-box attacks. The evaluation results of defense models under black-box attacks on CIFAR-10 and SVHN are reported in Table 3. Compared with the whitebox (PGD 20 ) results, all defense methods achieve much better robustness against black-box attacks. Compared with other models, ours is capable of generating stronger adversarial examples. Moreover, our model exhibits high robustness and natural accuracy on both datasets, demonstrating our method's excellent performance. For a fair comparison, we keep the training and evaluation settings the same as in MART [20] and HAT [6], and we ensure that the important training details involved in different methods are consistent with the original paper. Thus, we assert that MMAT is the closest to the optimal trade-off among these classical defense methods.
Discussion of MMAT's effectiveness. In Fig. 7, we collect the logit features of natural examples and find that our learned features have a larger distance between classes while being more clustered within the same class. The more distinguishable feature embedding justifies our improvement of both RA and NA. In particular, the distribution of MMAT is similar to that of SAT and MART, whose robustness is great, and MMAT can well distinguish ten classes' features of the dataset, like LBGAT, which has high natural accuracy. Then we study the distributions of examples' margins obtained by multiple models. Fig. 8 exhibits the distributions. We observe that the margins obtained by our models are consistently in the medium range of all models at to the decision boundaries of multiple models. We still use the models trained in Table 2, and calculate the margin of each model separately. Here we only randomly select 10,000 training examples. different margin levels, in line with our quest for moderate margins. In addition, the example size with small margins of SAT significantly exceed those of MMAT, while the example size with larger margins gradually less than us, which explains why our method can improve both robust accuracy and natural accuracy over SAT. Another interesting finding is that TRADES consistently has the lowest example size when the margin is small, and the highest example size as the margin gradually increases, which makes us understand better why it can stand out under the strong attacks of CW ∞ and AA.

Conclusion
Based on previous work and our experimental findings, we discover that adversarially trained models cause an excessive increase in the margin along adversarial directions. This partly contributes to the much-debated RA-NA trade-off. The commonly used uniform perturbation budget, , which ignores the characteristics of examples and leads to the severe issue of cross-over mixture, is a major contributor. To address the issue, our proposed MMAT considers the multiple characteristics of training examples and is fine-tuned by a teacher model to obtain a moderately inclusive decision boundary. Experiments show MMAT achieves a superior RA-NA trade-off compared to existing defenses.