{'fraction_non_alphanumeric': 0.0388021931674399, 'fraction_numerical': 0.011668775481512723, 'mean_word_length': 4.622011460185734, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
ODG-Q: Robust Quantization via Online Domain Generalization

Quantizing neural networks to low-bitwidth is important for model deployment on resource-limited edge hardware. Although a quantized network has a smaller model size and memory footprint, it is fragile to adversarial attacks. However, few methods study the robustness and training efficiency of quantized networks. To this end, we propose a new method by recasting robust quantization as an online domain generalization problem, termed ODG-Q, which generates diverse adversarial data at a low cost during training. ODG-Q consistently outperforms existing works against various adversarial attacks. For example, on CIFAR-10 dataset, ODG-Q achieves 49.2% average improvements under five common white-box attacks and 21.7% average improvements under five common black-box attacks, with a training cost similar to that of natural training (viz. without adversaries). To our best knowledge, this work is the first work that trains both quantized and binary neural networks on ImageNet that consistently improve robustness under different attacks. We also provide a theoretical insight of ODG-Q that accounts for the bound of model risk on attacked data.


I. INTRODUCTION
Despite the impressive performance of deep neural networks (DNNs) in various tasks [5], [27], [4], they are vulnerable to adversarial attacks [8]. A well-trained DNN can be easily fooled to misclassify by only a small (but carefully computed) perturbation on the input data that is even not noticeable to the human eyes. Such adversarial attacks can lead to serious security and safety concerns such as autonomous driving, mobile robots, face recognition, etc. On the other hand, quantization [1], [31], [28], [26] has become an essential step for DNN deployment, especially on resource-limited edge devices. Quantization converts the data format of weights and activation from floating-point (FP) to low-bitwidth integer (INT), thereby significantly reducing the computation and storage budgets.
Although quantization (or binarization in the extreme case) can largely reduce model deployment cost, a natural question arises: are quantized models robust enough to defend against adversarial attacks? According to [14], models are vulnerable under adversarial attacks when they are quantized to low bitwidths, e.g., ≤ 4 bits. When quantized to 1-bit, one gets binary neural networks [19], [15] that are extremely hardwarefriendly but their robustness against attacks, especially for largescale problems, remains underexplored. In addition, to improve model robustness, most previous defense methods [17], [7], [35] based on the FP models fall short to balance efficiency and robustness. For example, training with augmented adversarial samples offline can improve robustness but typically incurs 3× ∼ 30× overhead vs natural training [24], i.e., training on clean data without defense consideration. For example, Ref. [33] trains a robust classifier on ImageNet using 128 V100 cards at around 31× computational cost of natural training. Also, adversarial training often requires extra storage for adversarial samples, e.g., the PGD-K training [17] requires an outer loop for iterating the mini-batch data similar to natural training, plus an inner loop to generate K-step adversarial samples. Regularization-based method [23] enhances robustness by regularizing weights and/or activation, which does not require a huge training cost but their performance is often limited especially for large-scale datasets. AdvFree [24] uses 4 P100 GPUs and 2 days to train a FP model on ImageNet. AdvFree improves efficiency by recycling gradients that update the weights to generate adversarial perturbations. However, it still trains with each mini-batch for multiple loops per epoch to generate adversarial updates.
We argue that the holy grail is to generate diverse adversarial samples at a very low cost. Our main innovation is to recast the learning of adversarial robustness into a domain generalization problem. The natural samples are viewed as a subset of source data. Unlike previous defense methods [8], [17], [35] that only consider local adversarial perturbations generated from each sample itself in each batch, we argue that global adversarial perturbations in different batches are also important, which help the model to generate diverse adversarial samples. By constructing a set of adversarial perturbations and updating the subset of perturbations in turn during each iteration like a roulette, we obtain a bunch of adversarial samples from multiple source domains which have different times of adversarial updating. The adversarial perturbations are generated online so that different samples arrive in a batch-sequential order without an inner loop, thereby reducing the training time sharply and obviating the need to store adversarial samples (cf. Fig. 1).
To sum up, the contributions of this paper are threefold. 1) We recast the learning of adversarial robustness as a domain generalization problem. By generating adversarial source data from multiple domains online during training, and enforcing high-level feature alignment, the proposed algorithm achieves state-of-the-art robustness with a runtime close to that of natural training. 2) Inspired by the theory of domain generalization, we also analytically prove that the risk on target data (attacked samples on the inference model) can be bounded with the proposed approach. 3) Extensive experiments on MNIST, CIFAR-10 and ImageNet datasets demonstrate the effectiveness of the proposed ODG-Q, under various adversarial attacks in both white-box and black-box settings.

II. RELATED WORK AND PRELIMINARIES
A. Objective of Robustness Learning Definition 1. Given a natural dataset (x, y) (x: data, y: label) sampled from a domain D and a task loss function L Θ (·, ·), the goal of learning a robust model is to learn parameters Θ that maximize the performance under any perturbation p (within an -radius ∞ -ball) on x From Eqn. 1, we observe that optimizing the robustness of neural networks is a saddle point problem. It can be decomposed as an inner maximization problem to strengthen the deception ability of the attack given the perturbation range , and an outer minimization problem to minimize the expected error given any attack.

B. Representative Attacks
FGSM [8] is a fast attack method that performs a one-step update using the sign of the gradient of the loss function, which increases loss in its sharpest direction.
BIM [12] performs FGSM attack in multiple steps with a small step size α and clips each perturbation update into a given range .x PGD [17] is a variant of BIM that adds a random start. It projects the updated adversarial samples into an -radius ∞ball. PGD is proven to be a general first-order attack [17]. TPGD [34] is an improved version of PGD that considers robustness loss as the sum of the natural loss (classification loss) and the boundary loss through where ρ models the tradeoff between the two loss terms. These four attack schemes are employed in our experiments. On the defense side, adversarial training [17], [29] generates adversarial examples and then trains the model with them to improve robustness. Another branch of defense is randomization, which aims to randomize the adversarial perturbation into a random perturbation. Ref. [32] proposes a padding-and resize-based random input transformation to reduce adversarial perturbation. Random mask [16] filters the feature maps selectively with random masks, which helps a CNN capture spatial information and develop resistance to attacks.

C. Maximum Mean Discrepancy (MMD)
To bound the error in the target domain with assistance from source domains, MMD [21] has been extensively utilized as such a discrepancy metric.

Definition 2.
Let H be the Reproducing Kernel Hibert Space (RKHS) endowed with mapping function φ(·) : X → H. The MMD distance d M M D (·, ·) represents the distances between mean embeddings of feature generated by function φ(·). The expected MMD distance between two domains S and T is formalized by which usually utilizes induced kernel function k(x 1 , x 2 ) = φ(x 1 ), φ(x 2 ) H for MMD distance computation. In this work, the adopted kernel for φ(·) is the commonly used Gaussian continuous. MMD is employed to align the feature distribution between S and T in domain adaptation, which can be readily extended to domain generalization.
III. METHODOLOGY Since the adversarially augmented data introduced during the robustness training have a different distribution from the natural data, and attacked data (from target domain) is unknown during training, it makes sense to view robustness training as domain generalization. To this end, we recast the goal from a domain generalization viewpoint. Here we denote each clean datum as (x 0 , y) drawn from a natural source domain S 0 , and the augmented adversarial samples during training are drawn from other source domains S 1 , ..., S N k . Definition 3. Given (N k + 1) source domains, where natural data (x 0 , y) ∼ S 0 , and multiple adversarial data (x adv k , y) ∼ S k for k = 1, ..., N k . The joint distribution P i XY = P j XY for any 1 ≤ i = j ≤ N k + 1. The goal is to learn robust and quantized parameters Θ that achieve minimum performance error on unseen attacked target data (x t , y) ∼ T , i.e. generated during inference, where the target input data x t is generated by x t = argmax p L Θ (h(x 0 + p), y), p ∞ < . Both model parameters Θ and activations are limited to a small bitwidth (e.g., ≤ 4-bit).
In the following, the loss function L Θ (·, ·) is denoted as L for brevity. To improve the robustness of model against different attacks, a direct idea is to prepare diverse adversarial samples which have different times of adversarial attack iterations, different approaches of attacks, etc. Then adversarial samples can be combined with natural samples for training.
However, the aforementioned direct idea requires a huge space to store these adversarial samples, and expensive computation is needed for generating and training these adversarial samples. In addition, quantization/binarization introduces extra computation for all regular convolutional and fully-connected layers. To solve these problems, we propose that the adversarial data from multiple source domains S 1 , ..., S N k are generated and updated sequentially on-the-fly during training, instead of preparing them before training.
As shown in Fig. 2, representative defensive approaches suffer from low time-space efficiency due to the multiple iterations offline to generate adversarial training samples. However, the visibility of the whole adversarial training dataset may not be necessary to the model. Inspired by online learning [10], the main drawback of offline approaches can be overcome if the adversarial training data stream can be generated online continuously.
Intuitively, the adversarial samples should meet two requirements: 1) they should be generated at a low computational cost, therefore extra training loops should be avoided; 2) they should be diverse within the -radius ∞ -ball of clean data. Therefore, the gradient ascent information is considered in both global-level (inter-batch) perturbation and local-level (intrabatch) perturbation, but not local-level perturbation only like in previous schemes [8], [17], [35].
To this end, we use a global perturbation set P ∈ R N k ·B·C·H·W , where N k is the number of online adversarial domains, and the remaining dimensions have the same shape as a batch of input data. The global perturbation set P accumulates the inter-batch gradient information. To make the adversarial samples diverse for producing different adversarial updates, we only update P[k : N k −1, ...] per epoch, where k is the epoch index modulo N k . By doing so, P[0, ...] updates every N k epochs, and P[N k − 1, ...] updates every epoch like a roulette. The adversarial source datasets {(X adv k , Y ) ∼ S k |k = 1, ..., N k } are thereby constructed by incorporating global perturbation P[k : N k − 1, ...] and local perturbation on the natural dataset (X 0 , Y ) sequentially. P helps to recycle the gradient information from perturbations generation to weight updating.
The proposed training process is summarized in Algorithm 1 dubbed "ODG-Q" wherein we efficiently generate diverse adversarial samples online. For each batch, in Stage 1 (lines 4∼7), the global perturbation (inter-batch) in domain k is added to natural data to obtain x g k , then the local perturbation (intra-batch) is generated by gradient ascent on x g k . In Stage 2 (lines 8∼9), the global perturbation set is updated with the      current local perturbation, and the adversarial data x adv k is generated using global perturbation p k g and local perturbation p k l . In Stage 3 (lines 10∼11), the model is trained with x adv k . An MMD loss is added on the last convolution output for high-level feature alignment. Weight updates only once each batch without inner loop for adversarial samples generation. We halve the training epochs to keep our total training time consistent to natural training, since two backward passes are involved per batch in ODG-Q for perturbation generation and gradient descent, respectively. The theoretical insight of ODG-Q, which discusses how the risk on attacked data during inference (data from target domain) can be bounded by the proposed algorithm, is described in Appendix.

A. Experimental Results
Table I reports the model accuracy under both white-box and black-box attacks, together with training time for different methods on CIFAR-10. The "Natural" in the first row means experiments conducted on the clean natural data, and "Natural" in the method column means natural training without any defense approach. From Table I, we observe that low-bitwidth neural networks are sensitive to adversarial attacks for CIFAR-10 dataset. Even Gaussian noise on the natural data causes performance degradation from 87.5% to 72.9%. K-iterative Training can improve robustness. However, the expensive computational cost makes it hard to scale to large datasets. AdvFree(m = 4) and AdvFree(m = 6) are two variants of AdvFree that set 4 and 6 times inner loops, respectively. The results of AdvFree show that it may not improve robustness effectively in the low-bitwidth setting. DQ is a regularizationbased method having a small computational overhead, while its performance of robustness is not as good as training with the adversarial samples.
In contrast, our method 1) outperforms state-of-the-art methods in both accuracy and training time consistently by a large margin: achieving 49.2% average improvements under five white-box attacks and 21.7% under five black-box attacks in the setting of N k = 6.; 2) maintains good performance on     the natural data as well. The good generalization ability on the original natural data relies on the involvement of source feature during the whole training process. Therefore, we keep sending natural data and using MMD loss to align features between natural data and adversarial samples during training. From the comparison from N k = 1 to N k = 6 in our approach, the robustness against attacks is improved by introducing diverse adversarial source data. N k reflects the involvement frequency of global perturbation to generate adversarial source samples, apart from local perturbation. It shows the incorporation of multiple adversarial source domains can improve robustness very effectively. The performance on MNIST is reported in Table II. Our method achieves good accuracy under various attacks, with effi-cient training time per epoch. In Table III, we report the defense performance on the large-scale ImageNet dataset in both 4-bit and 1-bit settings. Due to the high computational requirement, few teams have attempted the adversarial robustness on the ImageNet dataset, let alone training in the low-bitwidth setting which also requires training extra quantization layers. To our best knowledge, this paper is the first to train both quantized and binary neural networks on ImageNet successfully against adversarial attacks. As shown in Table III, our method boosts the robustness to PGD, BIM and TPGD attacks steadily, with the total training time close to that of natural training. Since both the adopted quantization/binarization method and the model architecture (ResNet-18) in this paper are basic, there exists room for improvement for the ImageNet dataset.

A. Ablation Study of ODG-Q
We provide an ablation study of ODG-Q in Table IV. In the setting "w/o global pert.", we remove the global perturbation in Algorithm 1. The training is still in an online version that adversarial samples are generated dynamically during training. However, only local perturbation generated from the data itself can be used for adversarial training, which is also adopted in previous methods [24], [8], [17], [35]. The performance gap between "w/o global pert." and the full ODG-Q demonstrates the importance of global perturbation from different batch data, which gives the model a holistic view of data and helps the model to generate diverse adversarial samples in different source domains, i.e. the domain of source data depends on the times of using and updating global perturbation set during training. The visualization of different source data is shown in Fig. 3(a). The visualization shows that the data from different source domains have different gradient information.
In the remaining settings, we remove the MMD loss or set a small MMD loss coefficient. We can observe that MMD loss plays a significant role during training. Without the feature alignment between natural samples and various online adversarial samples by MMD loss, the training may become unconstrained.

B. Performance in Different Bitwidths
We further test the performance of our method when quantizing the network to different bitwidths. Although this paper focuses on robustness improvement for quantized and binary models, the proposed Algorithm 1 is orthogonal to the used bitwidth. Therefore, we also experiment on full-precision model (32-bit), besides low-precision settings ranging from 8-bit down to 1-bit. From Table V, our approach promotes robustness against different adversarial attacks by a large margin. The robustness improves significantly not only at full-precision settings, but more importantly, in quantized and binarized settings. We also observe that the level of improvement increases from 32-bit to 4-bit, and decreases from 4-bit to 1-bit. It indicates that the robustness improvement becomes a little easier when quantized to 8-bit, demonstrating that small adversarial perturbations can be absorbed in this case. However, in a low-bitwidth setting (≤ 4-bit), robustness learning becomes very challenging with a tightened bitwidth. It is reasonable since the capability of a model is strictly constrained by the low-bitwidth setting such that even training the model with diverse adversarial samples, the performance is limited.

C. Interpretability of Proposed Robust Training
In Fig. 3(a), we visualize what the model has learned from the source domain S 0 and online adversarial domains S adv1 and S adv2 . With small perturbations on the natural images, Grad-CAM [22] and Backpropagation Saliency map [25] are adopted to check the attention captured by the model. Interestingly, we find that there are fewer high-confidence areas that focus on the areas without the region of interest (RoI) in adversarial domains than that in the natural domain S 0 . For example, the Grad-CAM in S 0 treats not only the fishes themselves but also surrounding water areas as high-confidence areas, while the Grad-CAM in S adv1 and S adv2 tends to have lower interest in the surrounding water area. It hints that training with the online adversarial samples improves robustness by encouraging the model to focus more on the useful RoI.
We visualize the T-SNE projected features distribution of natural training, 5-PGD training and the proposed ODG-Q on CIFAR-10 dataset in Fig. 3(b). It is observed that the features of natural images (red) is quite different from the features of various attacked images, thereby the predictor using natural training can be deceived to give incorrect predictions on attacked images. Instead, by training natural domains and online adversarial domains altogether with MMD loss, the high-level features from different domains are aligned closer together, thus allowing the proposed model to give correct predictions regardless of the attacks.

VI. CONCLUSIONS
In this paper, we novelly recast robustness training as a domain generalization problem in an online version where multiple source data are generated dynamically. We then propose an strategy to generate diverse source data considering both global and local perturbations. Powered by the MMD loss, the training process is safely constrained that high-level features of the natural and adversarial online data are aligned, which reduces the expected risk of model on attacked data during inference in theory. Extensive experiments show that the proposed ODG-Q defends various attacks with a big margin over existing defense schemes in both white-box and black-box settings, using only minimal computation and a training time near natural training. We are hopeful that this work bridges the gap between network quantization and adversarial robustness.

APPENDIX
Regarding the generalization bound on the attacked target domain, a robust quantized model is supposed to constrict the upper bound of target risk. Here we discuss how the target risk can be bounded by the proposed algorithm.
According to the theory of domain adaptation in [2], [3], [20], let H be a RKHS with kernel function k(·, ·) induced by mapping function φ(·) : X → H. Then, the expected risk can be defined as the probability that a hypothesis h disagrees with any labeling function f within the domain D, Lemma 1. For any source domain S k with k ∈ [0, N k ] and the unseen target domain T , let F = {f ∈ H : ||f || H ≤ 1} be a hypothesis family inside a unit ball in RKHS H with a kernel k(·, ·). Assume the loss function L is convex with form |h(x) − f (x)| q for some q > 0, and obeys the triangle inequality, and bounded by ||L||| H q ≤ 1. The mapping φ(·) computed in the MMD distance is Lipschitz-continuous. Then, for every hypothesis h, we can always find a positive λ k where S 0 is the natural source domain and {S 1 , ..., S N k } are N k adversarial source domains.
where the input data x sampled from all domains {S 0 , ..., S N k , T } is around the corresponding natural data x 0 within a perturbation .
Eqn. 8 is very similar to the optimization objective proposed in ODG-Q (line 13), in which both the task loss (cross-entropy) on the adversarial samples from multiple online source domains and high-level feature alignment loss is minimized during training.

Proof for the Theoretical Insight
Proof of Lemma 1: By assuming ||L||| H q is bounded by 1, the expected risk on a domain can be measured in terms of the inner product in the corresponding RKHS [20]. Powered by the reproducing mapping property in RKHS, the expected risk on one domain D can be re-written as which holds for domain T and S k , k ∈ [0, N k ]. The ||L||| H q in bound assumption can also be safely extended by ||L||| H q ≤ n [18]. Mainly followed by the domain adaptation theory [20] of the relationship between source and target domain, we have We denote the natural data as x 0 ∼ S 0 , adversarial source data from k-th source domain x adv k ∼ S k , and attacked target data as x t ∼ T . Based on the perturbation assumption, every x adv k and x t is constricted in the -radius ∞ -ball around x 0 . Hence, there exists positive λ that sufficiently satisfies ||x k − x t || ≤ λ ||x k − x 0 ||. In addition, since the mapping φ(·) computed in MMD distance is Lipschitz-continuous, we can always find a positive λ k satisfying d M M D (S k , T ) ≤ λ k d M M D (S k , S 0 ) that bounds S 0 , S k and T in the RKHS. Combined with Eq.10, we have Proof of Theorem 1 Based on Lemma 1, by averaging the Eq.1 for any k ∈ {1, ..., N k }, and assigning λ = max {λ 1 , ..., λ N k }, we have From the equation above, the expected risk on the target domain (attacked data generated during inference) is highly related to the minimization objective of the proposed Algorithm in the main paper (line 10 of Algorithm 1). Since the target data are unseen during training, the MMD loss between the natural source domain and target domain cannot be directly employed. Fortunately, all the adversarial data are bounded within a certain range around the natural data. By bridging natural data and attacked data with diverse adversarial samples that have different times of adversarial updates, the expected risk on the target domain can be effectively reduced theoretically and empirically.
In the experiments for the CIFAR-10 and MNIST datasets, we adopt ResNet-20 as the network architecture. The batch size is set to 512 with an initial learning rate of 0.1. The total epochs for natural training and ours are set as 300 epochs and 150 epochs, respectively. The learning rate decays to one-tenth Fig. 4. Comparison of cross-entropy loss surfaces between natural training and ours (N k = 4) in 8-bit, 4-bit and 1-bit neural networks. We perturb one input image along random direction 1 and adversarial direction 2 . We can observe that 1) the loss scale is consistently small in our approach (amplitude reduces to 1); 2) in contrast to the sharp loss that increases along the adversarial direction in natural training, the loss surface in ours increases slowly along with two directions; 3) robustness training in 1-bit case becomes even more difficult than that in the 8-bit and 4-bit cases, expectedly. of the original every 50 and 100 epochs for natural training and ours. The coefficient λ is set to 3 by default, and is set to 0.003 for 1-bit experiment. For the attack methods GN, FGSM, PGD, BIM and TPGD involved in the experiments, we divide the clean images by 255 to scale them into the range [0, 1] before sending into the neural network, and then the magnitude of perturbation is also divided by 255. The perturbation alpha of small step size in PGD, BIM and TPGD are set as 4 with 20 iterations, which are used to generated PGD-20, BIM-20, TPGD-20 adversarial attacked data. We train experiments on CIFAR-10 and MNIST using one NVIDIA-3090 card. In the experiments for the ImageNet dataset, we adopt ResNet-18 as the network architecture. The total epochs for natural training and ours are set to 90 epochs and 45 epochs, respectively. The learning rate decays to one-tenth of the original every 30 and 15 epochs for natural training and ours. The schedule to generate attacked ImageNet data is consistent with that of CIFAR-10 and MNIST datasets. We train experiments on ImageNet with two NVIDIA-3090 cards.
In Fig. 4, we study the robustness against increasing disturbance in different bitwidth settings. We enlarge the perturbation gradually from 0 to 8, along the random direction In addition, the loss landscape is more rugged in the 1-bit setting, in which the model has only +1 or -1 representation after each binarized layer. For both weight and activation, the small perturbation may cause either error absorption or even the change of sign through each binarized layer, which leads to an uneven loss landscape. It illustrates why robustness improvement on the 1-bit network is the most challenging task, compared with other precision settings. Fig. 5 visualizes the MMD loss surface after training. The MMD distance shows the dissimilarity of the high-level feature distributions between the natural data and attacked data during inference. Large distance means dissimilar distribution. Compared with the loss surface with natural training (top line), the proposed approach enables MMD distance reduction effectively, which explains why the proposed approach classifies attacked data samples correctly, even having comparable performance with the model on clean data in some cases.
More visualizations about the generated online adversarial data in Fig. 6, where N k is the number of online adversarial domains. Both Grad-CAM and saliency map show that different gradient information is contained in data from different source domains.