{'fraction_non_alphanumeric': 0.03003232941977199, 'fraction_numerical': 0.015228858261017527, 'mean_word_length': 4.922573060127645, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Robust Transferable Feature Extractors: Learning to Defend Pre-Trained Networks Against White Box Adversaries

The widespread adoption of deep neural networks in computer vision applications has brought forth a significant interest in adversarial robustness. Existing research has shown that maliciously perturbed inputs specifically tailored for a given model (i.e., adversarial examples) can be successfully transferred to another independently trained model to induce prediction errors. Moreover, this property of adversarial examples has been attributed to features derived from predictive patterns in the data distribution. Thus, we are motivated to investigate the following question: Can adversarial defenses, like adversarial examples, be successfully transferred to other independently trained models? To this end, we propose a deep learning-based pre-processing mechanism, which we refer to as a robust transferable feature extractor (RTFE). After examining theoretical motivation and implications, we experimentally show that our method can provide adversarial robustness to multiple independently pre-trained classifiers that are otherwise ineffective against an adaptive white box adversary. Furthermore, we show that RTFEs can even provide one-shot adversarial robustness to models independently trained on different datasets.


Introduction
As deep neural networks have become increasingly common in security-conscious applications (e.g., autonomous driving or malware detection), protecting deployed networks from the manipulations of a malicious attacker has become increasingly important. It has been shown that neural networks trained to classify images using standard learning objectives are vulnerable to imperceptible pixel-level perturbations crafted by an intelligent adversary. The algorithms used to approximate maximally destructive perturbations are commonly referred to as adversarial attacks and are studied under formally defined threat models consisting of sets of assumed constraints on the adversary. While various attacks have been extensively studied under a variety of threat models, we focus exclusively on defending against a white box adversary, where the attacker has full knowledge of the model under attack and any defenses applied, as this is the most difficult scenario to defend against (Carlini et al. 2019).
Existing state-of-the-art defenses against white box adversaries induce robustness by training on adversarial examples generated throughout the learning process, a method often referred to as "adversarial training" (Madry et al. 2018). While the body of work surrounding adversarial training is rapidly expanding, there are several common drawbacks: (1) adversarial training is far more computationally expensive than the standard training paradigm (Madry et al. 2018); (2) adversarially trained models are known to be significantly less accurate on unattacked images than their non-robust counterparts (Su et al. 2018;Tsipras et al. 2019;Raghunathan et al. 2019); and (3) when compared to non-robust models, adversarially trained models are more dependent on model capacity to achieve improved performance, further increasing their cost (Madry et al. 2018). In this work, we explore the use of a deep neural network adversarially trained to be a reusable pre-processing defense against a white box adversary for the purpose of amortizing the costs of adversarial training over multiple non-robust models that are otherwise undefended.
We are motivated in part by previous studies that have shown that adversarial examples exploit two types of vulnerabilities: features and bugs (Ilyas et al. 2019;Nakkiran 2019). While bugs represent a unique vulnerability in an individual model, such as a poor decision boundary, features represent meaningful and predictive elements of a dataset that can be exploited under perturbation. Furthermore, the existence of feature vulnerabilities has motivated the transferability of adversarial examples (Ilyas et al. 2019). Thus, we hypothesize that this transferability property extends to defenses designed to correct for such vulnerabilities. More formally, we hypothesize that one can train a neural network to take as input an adversarially perturbed image and give as output a robust transferable representation with meaningful and predictive features that are useful across independently pre-trained and undefended classifiers. We refer to this network as a robust transferable feature extractor (RTFE), visualized in Figure 1.
While recent work has begun to investigate the usage of neural networks trained as pre-processing defenses (Liao et al. 2018;Zhou et al. 2021), the resulting models either: (1) are ineffective against a white box adversary, as shown in (Athalye, Carlini, and Wagner 2018); or (2) overfit to the model they are trained to defend, making them unable to successfully transfer, as shown in Section 4. In this work, we build from the theory of (Ilyas et al. 2019) and design a method to train an RTFE to defend independently pre-trained classifiers against a white box adversary. Furthermore, we investigate the transferability of the resulting RTFEs across both models and datasets. Finally, we build from the theory of (Nakkiran Figure 1: Our robust transferable feature extractor (RTFE) takes as input an adversarial example and gives as output a defended image that can be passed to independently pre-trained classifiers that are otherwise undefended. This reusable pre-processing defense is effective against an adaptive white box adversary even across neural architectures and datasets.
2019) by using model ensembling to more reliably exploit feature vulnerabilities for the purpose of increasing the transferability of RTFEs. To the best of our knowledge, we are the first to provide a transferable pre-processing defense that is effective against a white box adversary.

Background and Related Work
One of the most widely accepted defense methods against white box adversaries is the adversarial training approach of (Madry et al. 2018), of which there are numerous variations (Croce et al. 2021). Adversarial training replaces the standard learning objective with the robust optimization objective given by Eq. 1, which encourages a model, parameterized by θ, to minimize a loss function on all datapoints (x, y) in the training set D, while also being locally stable within a bounded neighborhood around each point.
Noticeably, this min-max formulation is computationally intractible; thus, the problem is often decoupled in practice. Standard approaches use projected gradient descent (PGD) to estimate the maximally worst-case perturbation δ of image x under a constraint of the form δ p ≤ , where · p denotes the p vector norm. The learned weights θ are then updated to minimize the standard learning objective on the perturbed images (Madry et al. 2018). While PGD generalizes across any choice of bounded norm, we primarily focus our attention on adversaries bounded by the ∞ -norm. Our adversarial examples are first initialized from a random uniform distribution, as given by Eq. 2. We then use Eq. 3 to iteratively generate adversarial examples, where α is the step size, and ∇x i L is the local gradient with respect to the input imagex i of the cross entropy loss L CE .
As an alternative to adversarial training methods, there have been many attempts to design pre-processing functions to defend against adversarial examples (Guo et al. 2018;Liao et al. 2018;Zhou et al. 2021;Joshi et al. 2022). (Guo et al. 2018) investigated a variety of traditional image preprocessing techniques (e.g., JPEG compression) for the purpose of removing malicious perturbations from adversarial examples; however, these techniques were shown to rely on a phenomenon referred to as gradient obfuscation that was later bypassed by (Athalye, Carlini, and Wagner 2018), demonstrating that these types of defenses are ineffective against a white box adversary. (Liao et al. 2018) train an autoencoder on a static dataset of adversarial examples extracted from an undefended pre-trained classifier. By viewing an adversarial example as an image corrupted by carefully tailored noise, they are motivated to train an autoencoder as an image denoiser for the purpose of removing adversarial perturbations. However,  show that this procedure also fails to provide a defense against an adversarial attack directed at the composition of the model and denoiser. (Zhou et al. 2021) train a denoiser to defend against a white box adversary by adversarially training a preprocessing autoencoder jointly with their classifier. We find that their method has two key limitations: (1) it requires a pre-trained robust classifier, which precludes the need for such a pre-processing defense; and (2) it does not provide robustness when re-used to defend a pre-trained classifier that was not adversarially trained, as further shown in Section 4.
Finally, independent of our research, (Joshi et al. 2022) study the impact of pre-processing defenses in the domain of automatic speech recognition. Among their set of experiments, they report favorable results for their pre-processing defense trained to provide adversarial robustness against a white box adversary for an otherwise undefended model; however, they do not explore the transferability of preprocessing defenses, which is the main focus of this work.

Robust Transferable Feature Extractors
Here, we detail our theoretical motivation and defense mechanism. In the following sections, we denote a natural image as x, its ground truth class as y, any adversarial perturbation as δ, and an adversarial example asx = x + δ. We represent a given neural network model with learnable parameters θ as f θ and an ensemble of N models as {f θi } N i=1 , where i ∈ {1, · · · , N }. Finally, We denote a robust transferable feature extractor (RTFE) with learnable parameters φ as g φ . (Ilyas et al. 2019) theorize that the success of adversarial examples can be attributed to two types of vulnerabilities:

Motivating Transferable Defenses
(1) "features", which are characteristics of images that represent useful, but unstable, directions for learning; and (2) "bugs", which are vulnerabilities or aberrations in the model corresponding to irrelevant (and sometimes detrimental) directions for learning. Formally, they define a feature as a quantifiable quality of an image, represented mathematically as any function f : X → R mapping the space of images X to the space of real numbers R. These features can be further sub-divided into two classes: vulnerable and robust. Whereas the correlation between vulnerable features and the true class of an image can be inverted under perturbation, the correlation between robust features and the true class of an image will remain the same (Ilyas et al. 2019). Whereas "bugs" represent unique vulnerabilities of an individual neural network, "features" are meaningful and predictive elements of the data distribution. Thus, the exploitation of vulnerable features motivates the success of transferable adversarial examples, as different models independently trained on the same dataset are likely to rely on similar sets of features to discriminate between images (Ilyas et al. 2019;Nakkiran 2019). Therefore, we hypothesize that this property also motivates the transferability of a pre-processing defense trained to either remove vulnerable features from an image or to extract robust ones. Furthermore, given that a pre-processing defense can provide protection against feature vulnerabilities for one model, it may also defend against these same vulnerabilities present in other models independently trained on similar datasets.
Consider a natural image x composed of both robust features (x robust ) and vulnerable features (x vulnerable ) such that x = I(x vulnerable , x robust ), where I(·) is some composite function that creates an image. Our goal is to train RTFE g φ to take image x as input and give as output a transferable representation free of vulnerable features such that g φ (x) ≈ x robust . Given that x robust contains meaningful and predictive elements that delineate the data distribution, g φ (x) should be able to provide adversarial robustness while maintaining predictive performance for independently pre-trained models that are otherwise undefended.

Learning a Transferable Defense Mechanism
To train an RTFE g φ , we build from the standard adversarial training procedure proposed by (Madry et al. 2018). However, rather than training a robust classifier to learn θ by minimizing the optimization objective given by Eq. 1, we adapt their method to learn φ by minimizing the optimization objective given by Eq. 4 using the logit loss given by Eq. 5 1 . We em- 1 We found the logit loss to be more performant when compared to the standard cross entropy loss proposed in (Madry et al. 2018). We also tested several auxiliary losses, such as the variations of the pixel loss from (Liao et al. 2018), but found that they did not impact performance. We omit them from our method for simplicity. ploy the Huber loss (Huber 1992), given in Eq. 6, as our logit loss due to the increased stability in the presence of extreme outliers, which are natural artifacts of adversarial training.
In our experiments, we use β = 1. To generate logit values during training, we use a frozen classifier f θ , which we refer to as a guide model, that is pre-trained using the standard learning objective.
(6) As discussed in Section 2, we decouple the min-max formulation of our objective given by Eq. 4 and iterate between: (1) generating adversarial imagex; and (2) minimizing the logit loss between f θ (g φ (x)) and f θ (x). To generate adversarial images, we use ∞ -PGD to maximize the cross entropy loss of the composite model f θ (g φ (x)). Note that while learning φ, we backpropagate gradients through the frozen classifier f θ without updating θ. Furthermore, our algorithm is agnostic to the selection of the pre-trained classifier, as further discussed in Section 4.
The network architecture of our RTFE is a modified UNet (Ronneberger, Fischer, and Brox 2015) based on the proposal of (Liao et al. 2018). We replace bilinear interpolation with nearest neighbor resize convolutions as this is known to reduce checkerboard artifacts (Odena, Dumoulin, and Olah 2016). Additionally, we replace the final layer with a pixel convolution followed by a sigmoid layer. Without this modification, the network frequently produced outputs with invalid pixel values. We also tested using the denoising autoencoder architecture from (Liao et al. 2018), but found it's performance on clean images was worse than that of the UNet while offering similar robustness.
One notable property is that RTFEs trained on a single model are not likely to distinguish between examples exploiting "bugs" from those exploiting "features", as both contribute to the guide network's vulnerability. Thus, the transferability of these RTFEs will be damaged as it is learning to protect against "bug" vulnerabilities that are not shared by other models. However, if the RTFE was instead trained on an ensemble of frozen guide models, we hypothesize it will better differentiate between "feature" and "bug" vulnerabilities, as the "features" are shared between the models it learns from, while the "bugs" are not. Thus, we invert the process proposed by (Nakkiran 2019), which creates adversarial examples that exclusively exploit bugs, to generate adversarial examples that are more likely to exploit features.

Robust Feature Extraction with Ensembles
In (Nakkiran 2019), the authors generate non-transferable adversarial examples using the attack objective given by Eq. 7. For an ensemble of N independently trained models each parameterized by θ i , this objective finds a perturbation that: (1) minimizes the cross entropy loss with respect to an incorrect target class y targ for model f ϑ ; and (2) minimizes the expected standard loss with respect to the correct class y over an ensemble of models {f θi } N i=1 not containing f ϑ . This has the effect of ignoring feature vulnerabilities while exploiting the bug vulnerabilities of model f ϑ , as bug vulnerabilities decrease the targeted adversarial loss with respect to y targ for model f ϑ without impacting the standard loss with respect to y for ensemble {f θi } N i=1 . To validate the assumption that the adversarial examples generated from this procedure do not exploit "feature" vulnerabilities, (Nakkiran 2019) demonstrate that their adversarial examples do not transfer to unique classifiers trained outside the ensemble.
To increase the transferability of an RTFE, we build from this procedure by reversing the logic to generate adversarial examples that are more likely to exploit "feature" vulnerabilities while avoiding "bug" vulnerabilities. To do so, we use Eq. 8 to maximize the standard loss with respect to y over ensemble {f θi } N i=1 . This has the effect of ignoring bugs, as the generated adversarial examples, by construction, induce erroneous predictions for all models in ensemble {f θi } N i=1 , and are thus not unique to any individual model. max Recall that we use the composition of RTFE g φ and guide model f θ to generate adversarial examples during training with the process depicted in Figure 2. To learn an RTFE using an ensemble of guide models, we replace guide model f θ with an ensemble of independently pre-trained classifiers {f θi } N i=1 , as depicted in Figure 3. Thus, for RTFE g φ we solve the optimization objective given by Eq. 9, where adversarial examplex is approximated using Eq. 8 from the composition of RTFE g φ and ensemble

Experiments and Results
We empirically evaluate the performance of our algorithm in three scenarios: (1) the standard white box threat model; (2) a white box cross-model transfer scenario; and (3) a white box cross-dataset transfer scenario. We compare our approach against adversarial training (AT) (Madry et al. 2018) and joint adversarial training (JAT) (Zhou et al. 2021), and omit comparisons to the approach of (Liao et al. 2018) as it is shown to be ineffective against a white box adversary (Athalye and Carlini 2018). Our models are trained to classify images using the CI-FAR10 and CIFAR100 datasets (Krizhevsky, Hinton et al. 2009), the most popular datasets for robustness research (Croce et al. 2021). When training our RTFEs, we maintain the hyperparameters reported in (Madry et al. 2018), aside from a shortened training time of 60 epochs as we found that training longer did not improve the robustness of our models. For an even comparison, we use a UNet autoencoder as the architecture for all defenses that require a pre-processing network. We use ResNet18 and MobileNetV2 as our pretrained classifiers and evaluate white box robustness using the projected gradient descent (PGD) attack with a step size of 1.25 255 for 20 steps (Madry et al. 2018). Furthermore, we evaluate at multiple values of such that the adversarial perturbation δ of the adversary is bounded by δ ∞ ≤ 255 , where ∈ {0, 2, 4, 8}. For convenience, we use = 0 to denote the performance on unattacked images. We also include evaluations with stronger attacks in Appendix A to confirm that our defense does not suffer from gradient obfuscation. All evaluations are performed on an AMD MI100 accelerator using PyTorch (Paszke et al. 2019). Additionally, we follow the best practices outlined in (Carlini et al. 2019) by: (1) backpropagating through any pre-processing mechanisms applied to the inputs of a given classifier, including the defense; and (2) reporting all successfully attacked images at some as being successfully attacked for all larger . Figure 4: We provide a random sample of clean input images x, the same images processed by an RTFE g φ (x), and the absolute pixel difference between the two.

White Box Pre-Processing Defense
First, we evaluate our algorithm as a pre-processing defense using the standard white box threat model. We train an RTFE using a frozen ResNet18 as our guide model. Recall that, in our algorithm, guide models are independently pre-trained on the target dataset under the standard learning objective. During evaluation, we consider the adversarially trained RTFE and the pre-trained ResNet18 as a composite model. For AT (Madry et al. 2018) and JAT (Zhou et al. 2021), we maintain the hyperparameter configurations reported in the original papers to the extent possible. We re-implement both algorithms and adversarially train both networks from scratch, which further ensures a fair comparison as different attack implementations and network architectures across frameworks could lead to inconsistencies. As an additional baseline, we independently evaluate the pre-trained and undefended ResNet18 that was used as the guide model to train our RTFE. We repeat each experiment 5 times using different random seeds and report the average top-1 test accuracy for both CIFAR10 and CIFAR100 in Table 1. In Figure 5, we visualize the model performance trends and provide standard deviation error bars as we increase the attack perturbation bound . In both Table 1 and Figure 5, we refer to the undefended ResNet18 as "None".
We observe that, even without updating the undefended guide model, our RTFE provides a defense against an adaptive white box adversary that is comparable to both AT and JAT. However, we observe that both JAT and AT have higher top-1 classification accuracy on unattacked images (i.e., = 0) on both datasets-a gap that is noticeably smaller on CIFAR100 than it is on CIFAR10. This is expected as both JAT and AT train all layers of the model to be robust against adversarial examples. In contrast, the model used to guide our RTFE is frozen during training such that only the RTFE is trained to be robust. Surprisingly, our method had the highest robust accuracy on CIFAR10 when = 8. We attribute this increase over AT to the additional model capacity resulting from prepending the RTFE to the ResNet18.  Table 1: We compare the performance of each defense against a white box adversary on both CIFAR10 and CIFAR100. We report the average top-1 classification accuracy over 5 independent runs.

Cross-Model Transfer Defense
Next, we evaluate our algorithm as a cross-model transfer defense under the standard white box threat model. To do so, we first train an RTFE using as our guide model a ResNet18 pretrained on CIFAR10. During evaluation, we use the resulting RTFE to defend two new models independently trained on the same dataset: (1) a new ResNet18 model trained from a different random initialization; and (2) a MobileNetV2 model. We compare the performance of our algorithm against JAT, where the pre-processing model was jointly trained on CI-FAR10 from scratch with a randomly initialized ResNet18 model. In this cross-model transfer defense experiment, we are unable to compare against AT as there is no transferable defense mechanism. We repeat each experiment 5 times using different random seeds and report the average top-1 classification accuracy in Table 2. Furthermore, we visualize the trends as we increase in Figure 6. We observe that our RTFE significantly outperforms JAT when used as a transfer defense for both classifiers for all > 0; however, we perform worse on unattacked images (i.e., = 0). We discuss how we use ensembling to improve performance on unattacked images in Section 4.5.

Cross-Dataset Transfer Defense
Here, we evaluate our algorithm as a cross-dataset transfer defense using the standard white box threat model. To do so, we first train RTFEs using as our guide models a ResNet18 and MobileNetV2 pre-trained on CIFAR100. During evaluation, we use the resulting RTFE to defend a model of the same architecture separately trained on CIFAR10. We again compare the performance of our RTFE algorithm against JAT, where the pre-processing model was jointly trained from scratch on CIFAR100 with a randomly initialized ResNet18 or MobileNetV2, respectively. We repeat each cross-dataset transfer defense experiment 5 times using different random seeds and report the maximum observed top-1 classification accuracy in Table 3. Furthermore, we visualize the trends in Figure 7 and provide standard deviation errors bars as we increase . As before, we see that our RTFE outperforms JAT on both classifiers for all > 0, yet performs worse on unattacked images (i.e., = 0). Surprisingly, we also observe that this cross-dataset transfer defense significantly outperforms cross-model with lower values of . We conjecture this can be attributed to the transferability of features shared between datasets and is in part due to the increased complexity of the source dataset when compared to the target dataset. We direct the interested reader to (Salman et al. 2020), which explores cross-dataset transferability within the scope of transfer learning.  Table 3: We compare the performance of our RTFE against JAT when transfered cross-dataset from CIFAR100 to CI-FAR10. We use ResNet18 (top) and MobileNetV2 (bottom) models trained on CIFAR100 as guides, before transfering to the same architectures trained on CIFAR10. We report the best top-1 test accuracy over 5 independent runs.

Impact of Model Size on Transferability
It is clear that network architecture has a significant impact on the performance of our RTFE when used as a cross-model defense. In Table 4, we report the top-1 test accuracy observed when transferring an RTFE trained and transfered to ResNet models of varied capacity. We repeat each experiment 3 times using different random seeds and report the average. When further investigating the impact of the guide model architecture on transferability, we observe that transfer defenses, unlike transfer attacks (Wu and Zhu 2020), do not perform worse when moving from a larger model to a smaller one. We conjecture that this is due to the impact of "bug" vulnerabilties, which asymetrically effect both transfer attacks and defenses. Whereas an RTFE may be affected by "bugs" in both the guide and defended models, a transfered adversarial example is only able to interact with "bugs" in the model used to generate it.

Improving Transferability with Ensembles
To examine our hypothesis that training our RTFE with an ensemble of guide models will improve transferability, we return to the cross-model transfer defense. Using our algorithm detailed in Section 3.3, we train our RTFE using an ensemble created by randomly selecting m models without replacement from a pool of n independently pre-trained ResNet18, Mo-bileNetV2, and SqueezeNet (Iandola et al. 2016) classifiers.
In our experiments, our pool consists of 2 ResNet18 mod-  els, 2 MobileNetV2 models, and 2 SqueezeNet models such that n = 6. During evaluation, we use the resulting RTFE to defend an independently trained VGG16 (Simonyan and Zisserman 2015). Note that this architecture is not included in the pool of ensembles to control for the impact of model similarity on transferability. Using a white box threat model, we evaluate the performance of our RTFE trained using increased ensemble sizes such that m ∈ {1, 2, 3, 4, 5, 6}. We repeat each experiment 5 times and visualize the trends in Figure 8, where we plot the mean and standard deviation as we increase . We also plot the performance of an adversarially trained VGG16 model as a baseline. When transferred to defend a model with a previously unseen network architecture, we observe that RTFEs trained using a single guide model (i.e., m = 1) were noticeably less performant than those trained with 2 or more guide models (i.e., m ≥ 2). Furthermore, we observe that, while the transfer performance on unattacked images ( = 0) linearly increases with the size of the ensemble, the transfer performance on attacked images ( = 8) saturates around an ensemble of 3. We conjecture this drastic increase in the performance of our RTFE on clean images can be attributed in part to being trained using more useful directions for learning (i.e., adversarial examples primarily exploiting feature vulnerabilities). However, bug vulnerabilities can still be exploited by malicious white box adversaries during inference. Thus, we do not see as strong a correlation between ensemble size and robust accuracy as with ensemble size and clean accuracy.

Conclusion and Future Work
In this work, we explored the use of transferable defense mechanisms against adaptive white box adversaries. We hypothesized that a deep neural network can take as input an adversarial example and give as output a robust representation that can be passed to independently pre-trained classifiers that are otherwise ineffective against a white box adverary.
Our results show that our robust transferable feature extractor (RTFE) increases the robustness of otherwise undefended classifiers in all scenarios investigated, thus validating this hypothesis. We summarize the following conclusions: • An RTFE trained using our algorithm introduced in Section 3.2 can be used as a pre-processing defense against a white box adversary to provide robustness comparable to adversarial training when used by its guide model. • Once trained, an RTFE can be successfully used as a cross-model transfer defense, where it provides adversarial robustness to another model independently pre-trained on the same dataset. • An RTFE can be successfully used as a cross-dataset transfer defense, where it provides adversarial robustness to another model independently pre-trained on a different dataset of similar or lesser complexity. • Training an RTFE using an ensemble of guide models increases its effectiveness as a cross-model transfer defense, where the performance on unattacked images increases linearly with ensemble size and the performance on attacked images saturates early.
Thus, once trained, RTFEs can be reused to defend models that have not yet been created and can amortize the costs of adversarial training over multiple non-robust models that are otherwise undefended. Several interesting directions remain open for exploration. In our experiments, we find that training to prioritize feature adversarial examples significantly outperforms training on a single classifier. It remains to be seen whether one can effectively utilize feature adversarial examples to improve regular adversarial training. Furthermore, our experiments show that RTFEs can be successfully reused across datasets for the same task. It may follow that one could reuse an RTFE across different tasks (e.g., image classification to object detection). We leave these investigations for future work.

A Gradient Obfuscation
To confirm that our robust transferable feature extractors (RT-FEs) do not suffer from gradient obfuscation, we follow the investigations of (Carlini et al. 2019). We start by noting that the UNet architecture used in our experiments does not contain randomized or non-differentiable layers; thus, gradient obfuscation is unlikely (Carlini et al. 2019). However, we confirm its absence by showing that: (1) single-step attacks perform worse than iterative attacks; and (2) incrementally increasing will inevitably cause model performance to fall below random. In these experiments, we use a more exhaustive projected gradient descent (PGD) attack to confirm that our choice of hyperparameters were valid for the experiments detailed in Section 4.
We compare the performance of three configurations of PGD in the standard white box scenario on CIFAR10: 1. A weak attack using a single step of PGD with α = 1.25 2. A strong attack that uses PGD with 100 steps, 5 restarts, and a step size of 1.25 100 3. The attack we used in Section 4.1, which uses 20 steps and stepsize 1.25 100 . All RTFEs are trained and tested on ResNet18 classifiers with the same weights used during training and testing. We evaluate at ∈ {0, 2, 4, 8, 16, 32} to show that the top-1 accuracy converges to random chance given a large enough ∞ bound on the adversary. Recall that we use = 0 to denote the performance on unattacked images. We visualize the model performance trends and provide standard deviation bars as we increase the . From examining Figure 9, it is apparent that the singlestep attack performs worse than both iterative approaches, and that at = 32 the model performs worse than a random classifier (i.e., top-1 accuracy less than 10%) on both iterative attacks. Additionally, increasing the number of steps from 20 to 100, and adding additional repeats, only marginally increased the attack success at = 8 indicating that our choice of hyperparameters was sufficient. Figure 9: We confirm that our RTFE does not suffer from gradient obfuscation by incrementally increasing . Using ResNet18 as our guide model for CIFAR10, we show that the top-1 accuracy converges to random chance with a large enough bound on the adversary.

B Evaluating RTFEs trained on Ensembles in the Standard White Box Threat Model
We demonstrate that the increased transferability of an RTFE trained on an ensemble is not an artifact of training a generally more performant RTFE. Consider two RTFEs g and g that provide 35% and 30% robustness, respectively, when transferred cross-model to defend against a bounded white box adversary. If g and g also provide non-transfer robustness of 65% and 40%, respectively, then it is unclear whether g is learning a more transferable representation, as the performance drop is significantly more severe than g when transferred. Thus, the natural question arises: do RTFEs trained on ensembles perform similarly to those trained on a single model while still being more transferable? We investigate this question with the following additional experimentation. We evaluate a defended ensemble in the standard white box threat model using an RTFE trained on CIFAR10 with an ensemble of 6 guide models under the same conditions detailed in Section 3.3. We then have that RTFE defend the same ensemble used during training. We compare the performance of RTFEs trained using a single model against RTFEs trained using an ensemble of models in both this non-transfer scenario as well as our cross-model transfer scenario. To evaluate the robustness, we use the same attack hyperparameters detailed in Section 4.1 with the loss function detailed in Eq. 8. In Figure 10, we visualize the performance trends in both the non-transfer and cross-model scenarios. For the cross-model experiment, we evaluate using a frozen VGG16 guide model as discussed in Section 4.5. We observe that, while RTFEs perform similarly on the model(s) used to guide training, there is a significant increase in robustness when used to guide novel models. This provides further evidence that training on adversarial examples that exploit feature vulnerabilities further increases the transferability of RTFEs.
(a) Non-Transfer (b) Cross-Model Figure 10: We compare RTFEs trained using a single guide model against those trained using an ensemble of guide models. We evaluate the performance of these RTFEs when used as either a non-transfer defense or a cross-model transfer defense on CIFAR10. We observe that our ensemble training algorithm increases transferability.