{'fraction_non_alphanumeric': 0.03793000965896426, 'fraction_numerical': 0.01333680065383758, 'mean_word_length': 4.733546325878594, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training

Deep neural networks (DNNs) are vulnerable to adversarial noise. A range of adversarial defense techniques have been proposed to mitigate the interference of adversarial noise, among which the input pre-processing methods are scalable and show great potential to safeguard DNNs. However, pre-processing methods may suffer from the robustness degradation effect, in which the defense reduces rather than improving the adversarial robustness of a target model in a white-box setting. A potential cause of this negative effect is that adversarial training examples are static and independent to the pre-processing model. To solve this problem, we investigate the influence of full adversarial examples which are crafted against the full model, and find they indeed have a positive impact on the robustness of defenses. Furthermore, we find that simply changing the adversarial training examples in pre-processing methods does not completely alleviate the robustness degradation effect. This is due to the adversarial risk of the pre-processed model being neglected, which is another cause of the robustness degradation effect. Motivated by above analyses, we propose a method called Joint Adversarial Training based Pre-processing (JATP) defense. Specifically, we formulate a feature similarity based adversarial risk for the pre-processing model by using full adversarial examples found in a feature space. Unlike standard adversarial training, we only update the pre-processing model, which prompts us to introduce a pixel-wise loss to improve its cross-model transferability. We then conduct a joint adversarial training on the pre-processing model to minimize this overall risk. Empirical results show that our method could effectively mitigate the robustness degradation effect across different target models in comparison to previous state-of-the-art approaches.


Introduction
Although deep neural networks (DNNs) achieve great success in applications such as computer vision [21,24,13], speech recognition [41] and natural language processing [38,10], they are found to be vulnerable to adversarial examples which are crafted by adding imperceptible but adversarial noise on natural examples [39,19,40]. Adversarial examples can remain destructive in the physical world [47,14] and transfer across different models [44,22]. The vulnerability of DNNs raises security concerns about their reliability in decision-critical deep learning applications, e.g., autonomous driving [15] and person recognition [49].
A major class of adversarial defenses pre-process adversarial examples to mitigate the interference of adversarial noise without modifying the target model [50,16,28]. Among pre-processing methods, We evaluate the white-box robustness (accuracy on white-box adaptive attacks) of three pre-processing defenses: APE-G [23], HGD [28] and NRP [32] on CIFAR-10 [26]. The target models are adversarially trained via two adversarial training strategies: Standard [31], and TRADES [52]. We combine the adaptive attack strategy with three attacks such as PGD [31], AA [9] and FWA [46] to craft adversarial examples. "None" denotes that no pre-processing defense is used. "Obl" denotes the pre-processing model trained using oblivious adversarial examples, and "Full" denotes the model trained using full adversarial examples.
compared with feature squeezing [20,50] and adversarial detection [30,34], input denoising [28,23,35,36,17,29,32] can remove adversarial noise and sufficiently retain the original information of the natural examples. Thus, input denoising methods are widely researched and they have shown great potential to safeguard DNNs from adversarial attacks.
However, many denoising methods only present security against oblivious attacks [3,40] where an attacker is completely unaware the defenses is being applied. Unfortunately, security against oblivious attacks is far from sufficient to be useful in practice. A serious attacker would certainly consider the possibility that a defense is used [3]. Recent researches [5,40] show that effectiveness of denoising methods are significantly weakened in an adaptive threat model where an attacker has full access to the pre-processing model or can estimate the knowledge (e.g., gradient) of the pre-processing model [6]. Especially, for a robust target model which is adversarially trained via state-of-the-art adversarial training strategies [31,11,42,52], empirical results as shown in Fig 1(a), 1(b) present that using the denoising defenses to pre-process inputs may significantly reduce rather than improve the adversarial robustness against worst-case (i.e., white-box) adaptive attacks. We name such adversarial robustness as white-box robustness, and this phenomenon as "robustness degradation effect".
A potential cause of the negative effect is that adversarial training examples are static and independent to the pre-processing model.  [18,25,8,37] have demonstrated that classic generative models are vulnerable to adversarial attacks. An attacker can mislead the pre-processing model to generate an output with respect to wrong class by disrupting the recovered example as much as possible. The pre-processing methods typically only focus on the ability of the pre-processing model to remove oblivious adversarial noise, but overlook the risk of the model being perturbed, which results in their insufficient adversarial robustness. To address this issue, we formulate an adversarial risk for the pre-processing model, to exploit the full adversarial examples to improve the inherent robustness of the pre-processing model instead of just using them to learn the denoising mapping. Corresponding to the above full adversarial examples found in feature space, we introduce a feature similarity term to measure the distance between natural and full adversarial examples. By this design, the adversarial risk is expected to reduce the distortion in both label space and feature space. The details of the adversarial risk are presented in Section 2.3.
Motivated by the above analyses, we propose a Joint Adversarial Training based Pre-processing (JATP) defense. Specifically, we use full adversarial examples found in the feature space as the supervision signal to train the pre-processing model. Then, we formulate a feature similarity based adversarial risk for the pre-processing model to improve its inherent robustness by using above full adversarial examples. Note that unlike the standard adversarial training which is model independent, our method only updates the parameters of the pre-processing model and needs to ensure that the parameters are suitable for different target models. This requires us to improve the crossmodel transferability. Considering that the natural examples used by different target models are consistent, and they have no adversarial patterns, we therefore introduce a pixel loss to reduce the distance between the pre-processed and natural examples. Based on the above designs, we conduct a joint adversarial training on the pre-processing model to minimize this overall risk in a dynamic manner. Experimental results in Section 4 show that our method effectively mitigates the robustness degradation effect against unseen types of white-box adaptive attacks in comparison to previous pre-processing defenses.
The main contributions in this paper are as follows: • We analyze two potential factors that cause the robustness degradation effect: (1) adversarial training examples used in pre-processing methods are independent to the pre-processing model; and (2) the inherent robustness of the pre-processing model is not sufficient due to its adversarial risk is neglected during the training process.
• we first formulate a feature similarity based adversarial risk for the pre-processing model to improve its inherent robustness by using adversarial examples crafted in a feature space against the full model. Then, we introduce a pixel-wise loss to improve the cross-model transferability of the pre-processing model. We propose a Joint Adversarial Training based Pre-processing (JATP) defense to minimize the overall risk in a dynamic manner.
• Experimentally, we demonstrate that JATP defense could significantly improve the white-box robustness of pre-processing defenses against adaptive attacks and mitigate the robustness degradation effect compared with the state-of-the-art. In addition, it could be applied to safeguard across different target models without additional training procedures.
The rest of this paper is organized as follows. In Section 2, we analyze the robustness degradation effect. In Section 3, we describe our defense method and present its implementation. Experimental results on different datasets are provided in Section 4. Finally, we conclude this paper in Section 5.
In addition, we briefly review related work on attacks and defenses in supplementary material A.

Analyzing the robustness degradation effect
In this section, we analyze two potential causes of the robustness degradation effect and explore to find their solutions.

Preliminaries
We first define the notation. We use bold lower-case letters (e.g., x, y) and lower-case letters (e.g., x, y) to denote vectors and scalars respectively. We use upper-case calligraphic symbols such as T and P to denote models.
In the setting of a K-class (K ≥ 2) classification problem, we are given a dataset target classification model with model parameters θ. The predicted class of an input example x is formulated as: where t k (x i , θ) is the logits output of the target model w.r.t. class k, and T k (x i , θ) approximates the probability (softmax on logits) that x i belongs to the k-th class, i.e., P (Y = k|X = x i ). We denote byx i ∈ R d an adversarial example, and by B ( Let P φ represent a pre-processing model with model parameter φ. Given an adversarial examplẽ is the output of the pre-processing model. We use F ψ to represent a full model with model parameter ψ. The full network F is composed of a pre-processing network P and a target network T . The class of an input adversarial examplex i predicted by F ψ can be represented by: is the logits output of the full model w.r.t. class k, and F k (x i , ψ) approximates the probability thatx i belongs to the k-th class. f k (x i , ψ) can be expressed as the output of a combined defended model: where ψ φ and ψ θ respectively denote the parameters for the pre-processing network and the classification network.

Absence of full adversarial examples
Pre-processing defenses typically first exploit one or several attacks to craft adversarial examples as adversarial training examples. These training attacks are usually only applied to the target model T θ without considering the pre-processing model P φ , and they thus are called as oblivious attacks [3,40]. Correspondingly, we call their adversarial examples as oblivious adversarial examples. An oblivious adversarial examplex i is crafted by solving the optimization problem: However, adaptive attacks tend to consider that a pre-processing defense model has been deployed. They can optimize a combined loss function on T θ (P φ (x i )) [40]   We consider two pre-processing defense methods, APE-G and HGD, which have great performances against oblivious attacks. The pre-processing models are trained by using the L ∞ PGD with the original training strategies in [23,28]. Given two robust target models which are adversarially trained via Standard [31] and TRADES [52], we evaluate the white-box robustness of the original pre-processing models (-obl) by using white-box adaptive attacks (see the blue bars in Fig 2(a)). The details of these test attacks can be found in Section 4.2. We then use same L ∞ PGD to craft full adversarial examples for retraining the pre-processing models.: where φ denotes the parameters of the retrained pre-processing models. We use the same loss functions as those in their original papers, and conduct an adversarial training with 40 iterations. The white-box robustness of the retrained pre-processing model is significantly better than that of original models (see the orange bars in Fig 2(a)). The results of this proof-concept experiment demonstrate that using full adversarial examples could improve the white-box robustness of the pre-processing defense. This discovery inspired us to use full adversarial examples to train the pre-processing model.

Vulnerability of the pre-processing model
Note that the results in Figure 2(a) also show that simply modifying the adversarial training examples in pre-processing methods does not completely alleviate the robustness degradation effect. This may be caused by the vulnerability of the pre-processing model. An attacker can mislead the preprocessing model to generate an output with respect to wrong class by disrupting the pre-processed example as much as possible. As shown in Fig 2( Since a white-box adaptive attack can exploit the knowledge of the full model to break the defense, we exploit the full adversarial examples to formulate an adversarial risk for the pre-processing model. In addition, Wang et al. [43] propose to explicitly differentiate the misclassified and correctly classified examples during the training, and they design a misclassification aware adversarial risk for improving the adversarial robustness of a target model. Inspired by their works, we formulate the adversarial risk of the pre-processing model as: where F ψ (x) = T ψ θ (P ψ φ (x)) is the class of the input x predicted by the full model F ψ , and x * i = arg maxx i∈B (xi) 1(F ψ (x i ) = y i ) denotes the full adversarial example. The 1(F ψ (x i ) = F ψ (x * i )) · 1(F ψ (x i ) = y i ) is the misclassification aware regularization term. Note that the target model T ψ θ is pre-trained and its model parameters ψ θ is fixed during the training process. According to this adversarial risk, we could design a method to train a pre-processing defense model with better inherent adversarial robustness.

Proposed method
Motivated by the analyses in the previous section, we propose a Joint Adversarial Training based Pre-processing (JATP) defense to mitigate the robustness degradation effect. We use full adversarial examples to train a pre-processing model that minimizes a hybrid loss composed of a pixel-wise loss and an adversarial loss. Fig 2.3 shows the visual illustration of our proposed defense.

Adversarial training examples
Based on the proof-concept experiment in Section 2.2, we apply adversarial attacks to the full model F ψ to craft full adversarial examples for training the pre-processing model P ψ φ . In addition, Kurakin et al. [27] point out that directly maximizing the cross-entropy loss in label space to craft adversarial training examples may lead to the label leakage problem. The problem allows a defense model to overfit on specific perturbations thus affecting model generalization to unseen attacks. Zhang and Wang [51] show that using a highly transferable attack to craft adversarial examples is beneficial to improve the generalization of the defense model, and finding the worst-case distortion in feature space can strength the transferability of an adversarial attack. Therefore, we redesign the adversarial training examples as follows: where f * (x, ψ) denotes the feature map of x on an internal layer of F ψ , and d(·) is a distance metric such as L 2 , cosine similarity or Wasserstein distance [2]. In this paper, we use the ResNet-18 [21] as the network of target model during training. Since the bottom layers tend to learn low-level features while the deeper ones are too specific to the label space, we choose the last convolution layer in the third basic block to obtain the feature map. We also study the influences of feature maps on different internal layers on the white-box robustness, and present their results in supplementary material B.

Loss function for JATP defense
We exploit a hybrid loss function to train our proposed JATP defense. The loss function is composed of two terms: pixel-wise loss and adversarial loss.
We study the influence of this pixel-wise loss on our proposed defense in Section 4.3, this loss help our pre-processing model retain the color and texture of natural examples, which can enhance the transferability of our defense across different target models.
Adversarial loss. As presented in Section 2.3, the adversarial risk of the pre-processing model consists of two parts: (1) the standard adversarial term 1(F ψ (x * i ) = y i ) and (2) the regularization , both of which have the indicator function.
In practice, it is intractable to directly optimize the 0-1 loss of the indicator function, and we thus use appropriate loss functions to replace the 0-1 loss.
For the standard adversarial term 1(F ψ (x * i ) = y i ), inspired by [43], we use a boosted cross-entropy (BCE) loss to replace the 0-1 loss, instead of the commonly used CE loss [31,28]. The BCE loss is defined as: Algorithm 1 JATP: Joint adversarial training based Pre-processing defense Input: A pre-trained target model T ψ θ , a pre-processing model P ψ φ , batch of natural examples x, perturbation budget , number of iterations T . 1: Initialization; 2: for i = 1 to T do 3: Craft full adversarial examplex * at the given perturbation budget using Eq. 5; 4: Forward-passx * through P ψ φ and calculate L pix using Eq. 6; 5: Forward-pass P ψ φ (x * ) through T ψ θ and calculate L adv using Eq. 9; 6: Back-pass and update P ψ φ to minimize L JATP using Eq. 10; 7: end for 8: return ψ θ .
where t k (·) is the probability output defined in Eq 1, and p(·) is the pre-processed output defined in Eq 2. ψ θ is the fixed parameters of the pre-trained target model T ψ θ . The first term is the commonly used cross-entropy loss, and the second term is used to reduce undesirable distortions generated by P ψ φ for decreasing the probability of the pre-processed examples with respect to the wrong classes.
For the regularization term, F ψ (x i ) = F ψ (x * i ) implies that adversarial examples have different output distribution to that of natural examples. Considering that the output is directly manipulated by the feature map on the internal layers of F ψ , and the full adversarial examples used to train the pre-processing model are found in the feature space, we thus use a feature similarity metric (FSM) to replace the first indicator function 1(F ψ (x i ) = F ψ (x * i )). The first term is expressed as: where d(·) is a mean square error metric, and t * (·) is the feature map on the internal layer of T ψ θ defined in Section 3.1. The another indicator function 1(F ψ (x i ) = y i ) is a condition that emphasizes learning on misclassified examples [43]. The misclassification based constrain used in adversarial training also help to enhance the inherent robustness of the pre-processing model. We use a soft decision scheme, i.e., the output probability 1 − t yi (p(x i , ψ ψ ), ψ θ ) to replace the indicator function 1(F ψ (x i ) = y i ). The overall adversarial loss of the pre-processing model can be defined as follows: where α is a positive hyperparameter.
The overall loss. Based on the pixel-wise loss and the adversarial loss, we present the overall loss for our proposed JATP defense: where β is a hyperparameter. The overall procedure is summarized in Algorithm 1.

Experiments
In this section, we first introduce the experimental setup used in this paper (Section 4.1). Then, we evaluate the adversarial robustness of our proposed JATP defense against unseen types of adaptive attacks (Section 4.2). Finally, we conduct the ablation and sensitivity studies to provide a further understanding of our defense method (Section 4.3).

Experiment setup
Datasets. We evaluate the adversarial robustness of pre-processing defenses on two popular benchmark datasets, i.e., SVHN [33] and CIFAR-10 [26]. SVHN and CIFAR-10 both have 10 classes of images, but the former contains 73,257 training images and 26,032 test images, and the latter contains 60,000 training images and 10,000 test images. All natural images are normalized into [0,1], and are performed simple data augmentations including 4-pixel padding with 32 × 32 random crop and random horizontal flip.  Table 2: White-box robustness (percentage) of pre-processing defenses on different target models. We show the most successful pre-processing defense with bold and the second one with underline.  [21]. Four adversarial training strategies are utilized to pre-train the target models, including Standard [31] MMA [11], TRADES [52] and MART [43]. Hyperparameters of these strategies are configured as per their original papers: λ = 6 for TRADES and MART. A PGD-10 attack with random start and step size 0.007 is utilized as the training attack. The perturbation budget is = 8/255 for both SVHN and CIFAR-10. The link to the implementation code of target models can be found in supplementary material C.
Defense Settings. We use three pre-processing defense methods as the baselines: APE-G [23], HGD [28] and NRP [32]. The architectures of the pre-processing model in APE-G and HGD are same as those in their original papers. We use the classification model adversarially trained by TRADES as the the target model during the training process. A PGD-40 attack with step size 0.007 is used as the training attack for APE-G and HGD. The perturbation budget is set to 8/255 for both SVHN and CIFAR-10. For NRP, we use the basic block in the original paper and only reduce the number of basic blocks to 3. The training strategies and other hyperparameters of baselines are consistent with the settings on CIFAR-10 in their original papers. For our method, we utilize the same architecture as NPR to build our pre-processing network. Our pre-processing model is trained using SGD [1] with momentum 0.9, weight decay 2 × 10 −4 and an initial learning rate of 10 −2 , which is divided by 10 at the 15 − th and 25 − th epoch (30 epochs in total). is set to 8/255 and the step size is 0.01. The hyperparameters α and β are set to 5.0 and 3.0 respectively. For fair comparison, all experiments are conduced on four NVIDIA RTX 2080 GPUs, and all methods are implemented by PyTorch.

Adversarial robustness evaluation
Robustness against adaptive attacks. We first evaluate the robustness of all pre-processing defense models against five types of attacks for both SVHN and CIFAR-10: PGD (40-step L ∞ PGD with step size 0.01), DLR (40-step L ∞ PGDDLR [9] with step size 0.007), AA (L ∞ -norm version of AutoAttack), FWA (20-step FWA with step size 0.01) and TI-BIM [12,48]. The perturbation budget is set to 8/255. All attacks exploit an adaptive attack strategy that the attacker have full access to the architectures and model parameters of both the pre-processing model and the target model. The white-box robustness of all defense models are shown in Table 1. Our proposed pre-processing defense JATP achieves the best robustness against all five types of attacks, and significantly mitigate the robustness degradation effect compared with other pre-processing models. Due to the space limitation, we present the results on SVHN in the supplementary material C. In addition, we use an adaptive attack strategy BPDA [4] to detect whether our work relies heavily on obfuscated gradients. We combine BPDA with L ∞ PGD to bypass the pre-processing defenses. As shown in Figure 4(a), our method achieves lower fooling rates than other baselines.  Figure 4: (a). Fooling rate (lower is better) of BPDA and PGD against pre-processing models. The target model is trained by TRADES. (b). Ablation study. We remove the pixel-wise loss ("Pix"), BCE adversarial loss ("BCE") and feature similarity adversarial loss ("FSM") respectively to investigate their impacts on our model.
Cross-model defense. We apply our proposed JATP defense to other three target models (i.e., Standard, MMA and MART) to evaluate its transferability. The white-box robustness of our proposed JATP defense on CIFAR-10 is reported in Table 2. We present the standard deviation in supplementary material C. Again, our proposed defense achieves higher robustness than other pre-processing defenses. Furthermore, we conduct an additional experiment under a relaxed constrain. That is, we update the parameters of the target model and the pr-processing model together during the training process, instead of using a pre-trained target model and fixing its model parameters. We then transfer the obtained pre-processing model denoted by JATP to above three target models. We find that the transferability of JATP defense has a slight improvement compared with JATP defense.

Ablation study
In this section, we conduct an ablation study to further understand the proposed JATP defense. We respectively remove the pixel-wise loss, BCE adversarial loss and feature similarity adversarial loss to investigate their impacts on our model. A target model trained by TRADES is used during the training process. As illustrated in Figure 4(b), removing pixel-wise loss mainly affects the transferability of the pre-processing model across different target models. Removing the BCE adversarial loss and the feature similarity adversarial loss would lead to a significant robustness degradation, which shows the adversarial loss is important to the pre-processing for improving its adversarial robustness.

Conclusion
In this paper, we analyze two potential causes of robustness degradation effect: (1) adversarial training examples used in pre-processing methods are independent to the pre-processing model; and (2) the adversarial risk of the pre-processing model is neglected during the training process. To solve this problem, we first formulate a feature similarity based adversarial risk for the pre-processing model to improve its inherent robustness by using full adversarial examples crafted in a feature space. We then introduce a pixel-wise loss to improve the cross-model transferability of the pre-processing model. Based on these, We propose a Joint Adversarial Training based Pre-processing (JATP) defense to minimize the overall risk in a dynamic manner. Experimental results show that our method effectively mitigates the robustness degradation effect against in comparison to previous pre-processing defenses. We hope that the work in this paper can provide some inspiration for future pre-processing defenses in improving the white-box robustness. The limitation of our work is that the our proposed method is only suitable for input denoising defenses so far, and we have not explored how to apply it to other pre-processing defenses. In the future, we plan to combine the pre-processing defense with recently proposed robustness models [45,7] and explore the potential improvements on the white-box robustness of pre-processing defenses.