{'fraction_non_alphanumeric': 0.036622938748619376, 'fraction_numerical': 0.013990350146297983, 'mean_word_length': 4.676198856137264, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Dispersed Pixel Perturbation-based Imperceptible Backdoor Trigger for Image Classifier Models

Typical deep neural network (DNN) backdoor attacks are based on triggers embedded in inputs. Existing imperceptible triggers are computationally expensive or low in attack success. In this paper, we propose a new backdoor trigger, which is easy to generate, imperceptible, and highly effective. The new trigger is a uniformly randomly generated three-dimensional (3D) binary pattern that can be horizontally and/or vertically repeated and mirrored and superposed onto three-channel images for training a backdoored DNN model. Dispersed throughout an image, the new trigger produces weak perturbation to individual pixels, but collectively holds a strong recognizable pattern to train and activate the backdoor of the DNN. We also analytically reveal that the trigger is increasingly effective with the improving resolution of the images. Experiments are conducted using the ResNet-18 and MLP models on the MNIST, CIFAR-10, and BTSR datasets. In terms of imperceptibility, the new trigger outperforms existing triggers, such as BadNets, Trojaned NN, and Hidden Backdoor, by over an order of magnitude. The new trigger achieves an almost 100% attack success rate, only reduces the classification accuracy by less than 0.7%-2.4%, and invalidates the state-of-the-art defense techniques.


I. INTRODUCTION
In recent years, deep neural network (DNN) has been increasingly widely used thanks to its excellent performance in object detection [1], voice conversion [2], and autonomous driving [3]. DNN has even outperformed humans in some tasks, e.g., large-scale image classification [4]- [6]. However, backdoor attacks put the applications of DNN to safety-or security-critical tasks at stake. Backdoored DNN was first introduced by BadNets [7], where a color block was used as a trigger and added on input images to encode its features into the parameters of the backdoored DNN in the training process. Many different triggers have been proposed since, including tatoo [8], graffiti [9], and cartoon patches [10].
To counteract backdoored DNNs, a range of defense methods have been developed. For example, STRong Intentional Perturbation (STRIP) [11] detects triggers by assuming a distinguishable entropy distribution of the DNN model prediction between clean and poisoned images. Februus [8] adopts Class Activation Map (CAM) [12] to visualize the decision Y. Wang  of a DNN, locate and remove the trigger, and restore the changed pixels using a pre-trained generative neural network (GAN) [13]. Spectral Signature Defense (SSD) [14] uses singular value decomposition (SVD) to obtain the dominant vector of samples in the training dataset and determine whether an input contains a trigger by comparing the correlation between the input and the vector against a pre-defined threshold. Activation Clustering (AC) [15] treats the trigger detection as a two-class clustering problem and utilizes K-Means to group the inputs into two clusters based on their hidden layer outputs. Other defense mechanisms include Artificial Brain Stimulation (ABS) [16], Neuron Pruning [17], and Neural Cleanse [18]. These defense approaches effectively defend existing triggers that typically produce intensive perturbations to a small region of an image (e.g., sunflower trigger [8]). A strong perturbation makes the triggers distinguishable in the final and/or intermediate output of a backdoored DNN, and makes the triggers perceptible.
Imperceptible triggers are more threatening to DNN applications and typically studied in the contexts of DNN adversarial attack [19], [20] and DNN robustness [21]. No backdoor needs to be installed in a DNN. Instead, a trigger is derived from the trained DNN to misclassify an image arbitrarily or into a target class. Imperceptibility was achieved by restricting the ∞ -or 2 -norm of the trigger. However, the triggers are input-specific, i.e., different images (even from the same class) require different triggers, leading to high complexities in trigger generation. The triggers can also be invalidated by changes in the images. Several input-agnostic adversarial trigger generation approaches have been developed [22], [23]. They exhibit much lower attack success rates (ASRs) than the input-specific imperceptible adversarial triggers. Moreover, a defender may have the knowledge of an attacker and potentially reproduce the triggers to reinforce the DNN through so-called adversarial training [20], [24].
In this paper, we propose a new backdoor trigger that is easy to generate, imperceptible, and highly effective. The new trigger is a uniformly randomly generated, three-dimensional (3D) binary pattern that can be horizontally and/or vertically repeated and mirrored and then superposed onto three-channel images for training a backdoored DNN model. While the new trigger collectively holds a strong recognizable pattern to effectively train or activate the backdoor of a DNN model, it generates weak perturbation to individual pixels and remains imperceptible. The complexity of the trigger generation and image perturbation is linear to the image size, and substantially lower than those of the existing triggers. Extensive experi-arXiv:2208.09336v1 [cs.CV] 19 Aug 2022 ments demonstrate the superiority of the new trigger to existing methods in complexity, imperceptibility, and effectiveness.
• Low complexity: The generation of the new trigger only involves linear operations, such as repetition, addition, and clamping (to within [0, 255]). The complexity is O(CW H) for an image with C color channels and W × H pixels per channel. In contrast, the existing imperceptible triggers, e.g., AdvGAN [22], UAT [23], and Hidden Backdoor [25], require back propagation and gradient descent operations on a neural network with dramatically higher complexities. The generation of a malicious input using the new trigger is computationally efficient, since the trigger is input-agnostic and can be directly superposed onto a benign image to perturb the image. In contrast, existing approaches, such as projected gradient descent (PGD) [24] and DeepFool [26], produce input-dependent triggers and perturb inputs using the gradient of the attacked neural network. • Imperceptibility: Compared to existing backdoor triggers typically using more visible colored blocks (e.g., [7]- [10]), our trigger is imperceptible. The imperceptibility of the new trigger is analyzed both quantitatively and visually on three different datasets. We show that the proposed trigger outperforms all baseline approaches in two popular perceptual metrics, namely, Structural Similarity (SSIM) Index [27] and Learned Perceptual Image Patch Similarity (LPIPS) [28], and in the visual difference between clean and poisoned images. • High effectiveness: The new trigger is evaluated experimentally under different datasets, neural network architectures, and latest defense strategies. We show that the trigger threatens the reliability of image classifier models with an ASR of close-to-100%, which is 17% higher than the existing input-agnostic imperceptible triggers, such as AdvGAN [22] and UAT [23]. The new trigger also invalidates the existing defense strategies. The rest of this paper is organized as follows. Section II provides the related works. Section III presents the threat model and the new trigger. The new trigger is extensively tested against popular datasets, DNN models, and defense strategies in Section IV, followed by conclusions in Section V.

A. Backdoor Trigger
Recently, several backdoor triggers have been proposed, mostly visible to human eyes, such as a small patch beside a digit [18], a pair of bright-rim eyeglasses on a face [29], a bright pixel in an image [30], or a cartoon watermark [11]. BadNets [7] uses a small patch (e.g., a yellow rectangle sticker on a traffic sign) as a trigger to poison images and trains a neural network model with the poisoned images.
Invisible backdoor triggers have been designed in [25], [31]- [34]. Saha et al. [25] introduced Hidden Backdoor (HB) attacks, which used a small image patch as a trigger. An image with invisible perturbations was produced by minimizing the difference of neuron activations in the penultimate layer between a clean image and its poisoned version with the trigger. However, once the backdoored DNN is deployed, an attack still requires a visible trigger to activate the embedded backdoor. Backdoor defense strategies, such as Februus [8], can detect and eliminate the trigger.
Li et al. [31] designed two types of invisible triggers for backdoor attacks. The first type was produced by converting a static trigger (for example, a string of texts) to a binary form and then replacing the least significant bits of pixels in an image with the trigger. Only altering the least significant bits results in minor changes in color intensity, making a trigger hard to detect by human inspectors. However, a defender can disable the trigger by replacing the least significant bits with random values. The second type was to amplify the triggerinduced activations of a subset of neurons in the DNN's penultimate layer. The process also reduced the trigger's norm to below a threshold, making the trigger imperceptible. Hence, a backdoor was implanted in a small number of neurons. In the presence of the trigger, those neurons can produce significantly larger activations than the others, making them detectable for detection strategies, e.g., AC [15], and suppressible using neuron pruning techniques, e.g., Neural Cleanse [18].
Zhong et al. [32] developed two invisible triggers. One was a static perturbation mask based on a repeated pattern. The pattern consists of an array of small sub-regions, and increases the color intensity of pixels in the sub-regions. Since this trigger is not random, it can be reverse-engineered by enumerating all possible sizes of the sub-regions and the increase of color intensity. The second trigger employed the DeepFool [26] to obtain a universal, invisible perturbation by projecting the images of one class to the boundary between their source class and the adversary-specified target class. The adversary can poison the training data, insert the backdoor into the DNN model, and use the perturbation as a trigger. However, the trigger depends on the source and target classes and is computationally expensive to produce.
Trojaned NN [33] and RobNet [34] are two recent and related designs of backdoor triggers. Trojaned NN [33] selects the most connected neurons in the penultimate layer of a pre-trained neural network model, and generates a trigger that maximizes the activation of the selected neurons using gradient descent. A backdoored model is obtained by further training the model with images poisoned with the trigger. RobNet [34] is a variation of Trojaned NN, and conducts neuron selection and trigger generation. It supports multiple trigger locations (up to eight per image) and multiple triggers (at different locations of an image) to produce a backdoored model. AdvGAN [22] and UAT [23] are two other recent designs of imperceptible triggers. AdvGAN adopts GANs to train a perturbation generator. For any image in the same domain as the images used to train the generator, the generator produces and adds a trigger to the image to attack a DNN model. UAT produces an input-agnostic imperceptible trigger for an image dataset and a given DNN architecture. It solves an optimization problem of universal perturbation by adopting the stochastic gradient method. AdvGAN and UAT require back propagation and gradient descent operations on a neural network, incurring high complexities.

B. Defense Methods
Defense strategies have been developed to detect or disable triggers, or repair backdoored DNN models [8], [11], [14]- [18], [35]. Februus [8] sanitizes inputs by removing potential trigger artifacts and keeping the information for classification tasks. The triggers are located using GradCAM [12], a variant of the classic DNN visualization technique, CAM [36]. It was reported in [8] that Februus reduced the ASR from 100% to nearly 0% for a badge, tattoo, image patch, and color block triggers on the CIFAR-10, German Traffic Sign Recognition Benchmark (GTSRB), Belgium Traffic Sign Recognition (BTSR), and VGGFace2 datasets.
STRIP [11] is a trigger detection algorithm for vision systems. It perturbs the input to a DNN model by superimposing various image patterns, and then observes the randomness of predicted classes for the perturbed inputs. The entropy exceeding a pre-defined threshold in predicted classes indicates the presence of a backdoor trigger. STRIP archives close-tozero false acceptance rate and false rejection rate on small black square, heart-shape frame, mosaic patch on MNIST, CIFAR-10, and GTSRB datasets. The effectiveness of STRIP depends on the existence and selection of the threshold.
SSD [14] detects triggers by first calculating the covariance matrix of the feature representation of training samples for each class. Then, SSD calculates the correlation between the feature representation of the incoming input and the top eigenvector of the covariance matrix (i.e., the eigenvector corresponding to the largest eigenvalue). It compares the correlation with a predefined threshold to detect triggers. Let denote the ratio of poisoned data in the testing data. SSD is effective under an -spectrally separable condition, i.e., (1) where W and D are the distributions of the inner representations of the poisoned and clean samples in the hidden layer; F = W +(1− )D is the mixture of W and D; µ F is the mean of F; v is the top eigenvector of the covariance of F; and ξ is the threshold to distinguish clean and poisoned samples. AC [15] adopts K-Means to cluster input images into clean and poisoned groups by inspecting the activations of the hidden layers of a DNN model. AC first retrieves the activations, then reduces their dimensions with primary component analysis (PCA) [37], fast independent component analysis (Fast ICA) [38], or t-distributed stochastic neighbor embedding (t-SNE) [39]. AC determines the clusters containing poisoned samples using one of four cluster analysis methods: Smaller, Relative Size, Distance, and Silhouette. The cluster with the fewest items is selected as poisoned by Smaller. Relative Size classifies a cluster as poisoned if the smaller one contains less data than a threshold. Distance classifies a cluster as poisoned if its median activation is closer to the median of another class than to its own. Silhouette analyzes the suspicion level based on size and Silhouette score [40].
Neuron Pruning [17] is under the premise that the average activation of neurons in the final convolutional layer of a backdoored neural network is significantly different between clean and adversarial inputs. The neurons that are dormant for clean inputs are removed to disable the backdoor. The termination condition is that the decrease of the classification accuracy on clean inputs exceeds a threshold, e.g., 4%.
Neural Cleanse [18] is a trigger detection and mitigation method for DNN backdoor attacks. It detects and reverseengineers a trigger by finding the minimal trigger required to misclassify all samples from other labels to the target label. The trigger size is measured by the number of pixels replaced. The mitigation techniques include input filtering, neuron pruning, and unlearning. Input filtering discards inputs with potential triggers. Neuron pruning removes backdoorrelated neurons identified by the reverse-engineered trigger in the penultimate layer of the DNN model. Unlearning trains the backdoored DNN to forget the trigger by using the reverseengineered trigger and correct labels.
Spatial Smoothing [35] is a class of widely used techniques in image processing for suppressing image noise. Local smoothing methods make use of nearby pixels to smooth each pixel. By selecting different weighting mechanisms for neighboring pixels, a local smoothing method can be Gaussian, mean, or median smoothing. The median filter runs a sliding window over each pixel, where the center pixel is replaced by the median value of the neighboring pixels within the window. Spatial Smoothing is effective in defending malicious inputs having invisible triggers, such as adversarial examples generated by PGD [20].
Last but not least, adaptive attacks can minimize the difference of neuron activations within each layer or reduce the difference of neuron activations between benign and malicious inputs (e.g., by minimum or min-max criteria) during data poisoning. ABS [16] provides an effective means to defend adaptive attacks by defining an adaptive loss function and minimizing it together with the classification loss function.
As will be shown in Section IV-D, none of the above stateof-the-art techniques is effective in detecting or disabling the new backdoor trigger discovered in this paper.

III. NEW BACKDOOR TRIGGER TO IMAGE CLASSIFICATION NEURAL NETWORKS
In this section, we first describe the threat model of the new backdoor trigger. Then, we describe the trigger generation, followed by implementation considerations.

A. Threat Model
We adopt a threat model similar to the one used in [34], where a cloud service provider delivers Machine-Learningas-a-Service (MLaaS) Platform. DNN training usually requires significant computing resources, e.g., GPU clusters, and domain knowledge of the DNN design and hyperparameter configuration. Commercially available MLaaS platforms, e.g., AWS Machine Learning, Google Cloud Machine Learning, and Microsoft Azure ML Studio, are widely accessible for DNN users to outsource DNN model training. In this case, the cloud service provider can be adversarial (because of a rogue employee or a compromised server). A user outsourcing the DNN training to the cloud provider is the defender. The DNN user evaluates the prediction performance of the received DNN model with clean datasets (e.g., images). The user may also try to detect the backdoors in the DNN model, and disable backdoor triggers using the state-of-the-art algorithms.
The goal of an adversary performing a backdoored DNN attack can be formulated as the following Maximum Likelihood Estimation (MLE) problem: where • c t is the target class specified by the attacker; • θ is the set of the model parameters of the DNN, i.e., the weights of connections and the biases of neurons; The adversary trains the DNN model to misclassify any poisoned input (i.e., images embedded with the trigger T) to the target class specified by the adversary. For clean inputs (e.g., images without the trigger T), the adversary wishes the DNN model to provide a satisfactory classification accuracy. The joint optimization can be readily decoupled into the separate design and optimization of the trigger T and the model parameter θ. Specifically, T can be viewed as an additional feature of the images, and is used to classify the images into a target class specified by the attacker, as done in existing backdoored models, e.g., BadNets [7]. According to the Universal Approximation Theorem [41], a neural network can approximate any continuous function at any given precision requirement. In this sense, given any trigger T, the model parameter θ can be trained to achieve any pre-specified classification accuracy of clean images and any pre-specified misclassification rate of poisoned images. For this reason, the attacker could design T, prior to the training of θ based on the trigger T, as done in BadNets [7].

B. The New Backdoor Trigger
We discover a new backdoor trigger T, which is a randomly generated binary 3D matrix superposed to the pixels of the input image. A DNN trained with poisoned images, X , can detect the presence of T and, therefore, contain a backdoor that can be exploited in a later stage. By carefully configuring its magnitude m, the backdoor trigger T can be imperceptible to human eyes. T is also input-agnostic, since it is generated independently of input images.
Area to be perturbed Step 1 Step 2 Fig. 1: The generation process of the proposed trigger, where a red box corresponds to a "+m" and a blue box corresponds to a "−m".
1) Generation of the New Trigger: As illustrated in Fig. 1, the new trigger is generated in three steps.
Step 1: Produce a 3D random matrix with the dimension of where each layer comprising the first two dimensions of the matrix corresponds to one channel of the images to be poisoned. The third dimension indicates different channels of the images. The elements of the 3D matrix follow the i.i.d. binary distribution with amplitude m (i.e., "±m") and are generated using the Cryptographically Secure Pseudo-Random Number Generator (CSPRNG).
Step 2: Extend each layer of the 3D matrix using repetition, i.e., repeating every element horizontally and vertically multiple times on each layer. Step 3: Mirror the 3D matrix horizontally (and/or vertically) on each layer to produce a horizontally (and/or vertically) symmetric matrix used as the new trigger. When poisoning an image, each page of the trigger is superposed (i.e., added) onto the middle of the corresponding channel of the image with margins reserved unperturbed to bypass Februus trigger removal [8]. The resulting magnitude of each pixel is truncated to be within the valid range of [0, 255] N C . The purposes of the repetition in Step 2 and the symmetric extension in Step 3 are to get around the typical image blurring method, such as Spatial Smoothing [35], and image transformation, such as flipping.
The numbers of horizontal and vertical repetitions per pixel in Step 2, denoted by R H and R V , and the width of the unperturbed margin, denoted by M G , are hyperparameters and can be adjusted, given the size of the 3D random matrix generated in Step 1, i.e., T H × T V × N C , and the size of the images to be poisoned, i.e., L H × L V × N C . Apparently, Fig. 1.
A backdoor trigger is a secret withheld by an attacker, and can be constructed from a random sequence generated by a pseudo-random number generator. Most statistical Pseudo-Random Number Generators (PRNGs) are based on recurrences and can be recognizable by assessing output streams [42]. It is possible to predict the future output of PRNGs [43] based on past observations. In contrast, a CSPRNG avoids detectable regularities and can withstand cryptanalysis conducted by a defender with full knowledge of the CSPRNG used. Given a sequence of pseudo-random bits generated by a randomly initialized CSPRNG, it is impossible to predict the next bit with a probability greater than 1/2 using a probabilistic polynomial-time algorithm. The proposed trigger is much easier to generate than the existing imperceptible triggers. The generation of the new trigger only involves linear operations, such as repetition, addition, and clamping (to within [0, 255]). It incurs the computational complexity of O(CW H) for an image with C color channels and W × H pixels per channel. In contrast, the existing imperceptible triggers, such as AdvGAN [22], UAT [23], and Hidden Backdoor [25], require back propagation and gradient descent operations on a neural network. Their computational complexities are significantly higher than O(CW H).
2) Selection of Trigger's Magnitude: We analyze the required magnitude of the proposed backdoor trigger for specific ASRs. Our analysis based on a Perceptron model suggests that the new trigger is not only feasible, but effective in creating a backdoor and poisoning the model.
Let P denote a 3D random matrix generated using the CSPRNG. For illustration convenience, we assume that the size of P, i.e., T H × T V × N C , is the same as the size of an input image X, i.e., L H × L V × N C . Then, the new trigger is T = mP, where m ∈ R + is the magnitude of the trigger. The size of the trigger is Considering the Perceptron model in Fig. 2, which takes vectorized images as the input. The magnitude of the trigger, m, is obtained by solving the following problem: where p and x are the vectorizations of P and X; T (x + mp) is the poisoned version of x, see (3); and η is the ASR.
Since the elements in p are i.i.d., E[x T p] = 0 and p T p = M . By vectorizing T to t, we set w = t = mp and b = − m 2 M 2 in (4). Then, where x i and t i are the i-th elements of x and t, respectively. Assume that the pixels are independent in the image x. Then, x t obeys the Gaussian distribution according to the Law of Large Numbers. Also assume that where As a result, The rate of the Perceptron recognizing the backdoor trigger in Fig. 2 (i.e., the ASR) is given by where Q(·) is the tail distribution function of the Normal distribution. A large value of m increases the ASR (at an increased risk of the trigger being perceptible to human eyes). Given the ASR η, the magnitude m satisfies where Q −1 (·) is the inverse function of Q(·). The right-hand side of (9) provides a lower bound for m, since w and b in (4) are designed only for trigger recognition. The lower bound confirms that a higher-resolution image is more susceptible to backdoor attacks since the image can accommodate a longer and less visible trigger (with a larger M and a smaller m). To this end, the new trigger may be less effective when smallsized, black and white images are perturbed. This is because the trigger can be more visible on black and white images, especially when the images are small and the amplitude of the trigger needs to be large to be effective. As discussed in Section III-B1, repeating each randomly generated ±m element horizontally and vertically (over R H and R V pixels, respectively) and then mirroring horizontally (and/or vertically) on each layer of the 3D matrix to produce a horizontally (and/or vertically) symmetric 3D trigger, are recommended to get around Februus trigger removal [8] and image transformation-based trigger disabling [44]. In the experiments presented in Section IV, we repeat every random element horizontally and vertically and then mirror it horizontally to produce a horizontally symmetric trigger. While the trigger size is M , the number of randomly generated elements in the trigger is M/(2R H R V ) (since each element is replicated for 2R H R V times, one per pixel). The magnitude The magnitude needs to be distributed evenly among the 3) Implementation Consideration: Defenders may apply image transformation techniques (e.g., random cropping, flipping, or rotation) to disable a trigger. To bypass these defenses, the trigger can be generated to be horizontally and vertically symmetric. Data augmentation can also be applied when training the backdoored DNN, by generating randomly cropped and rotated versions of poisoned images. Defenders may detect triggers by observing the contribution of pixels to classification through DNN visualization techniques, such as CAM [12]. A smaller-sized trigger can be generated only to perturb the inner part of an image. The margins remain intact to escape the inspection of activation in the margins.
According to the moments of the distribution of input images, the images are often normalized in DNN training. To prevent the normalization of input images from destroying the trigger, the attacker can unnormalize the trigger T before applying it to an image. The unnormalization can be given byT = T × σ + µ, whereT is the unnormalized trigger; µ and σ are the mean and standard deviation of the input images, respectively. Since µ and σ are publicly known, the unnormalization can be readily accomplished by the adversary.

IV. EXPERIMENT RESULTS
In this section, we experimentally evaluate the proposed backdoor trigger in terms of the attack capability and its resilience toward popular defense strategies.
1) DNN Model: We evaluate the threat of the new backdoor trigger by considering a customized Multilayer Perceptron (MLP) and a Convolutional Neural Network (CNN).
• MLP: We consider a 7-layer MLP made up of three fullyconnected layers, two Leaky ReLU activation layers with a negative slope of 0.2, and two Dropout Layers with the dropout probability of 0.2; see [45]; • CNN: We choose the 18-layer ResNet, where, besides the heading convolution layer and the fully-connected layer at the end of the CNN, the remaining sixteen convolutional layers in the middle are grouped into eight pairs. In each pair, a skip connection adds the input of the first convolutional layer to the output of the second convolutional layer; see [46]. 2) Dataset: The following public datasets produce poisoned data for training and testing backdoored DNNs. In each training dataset, 5% of images are poisoned with the new backdoor trigger. We train all three datasets with 100 epochs using a Stochastic Gradient Decent optimizer with the learning rate of 0.1, momentum 0.9, and weight decay 5 × 10 −4 .
• MNIST [47]: MNIST is a handwritten digital dataset, consisting of 60,000 training and 10,000 testing samples. Each sample is a 28 × 28 gray-scale image. Any class in the dataset can be chosen as the target class. We choose five as the target class. The trigger used is horizontally  Table I   . This metric quantifies the trigger detection accuracy of the considered defense algorithms. We adopt two perceptual metrics to quantify the imperceptibility of a trigger in a poisoned image: • Structural Similarity (SSIM) Index [27], an extensively adopted metric for measuring the structural similarity of two images; • Learned Perceptual Image Patch Similarity (LPIPS) [28], a metric designed specifically to quantify the invisibility of triggers in poisoned images.
A. Attack Success Rate The attacking performance of the new trigger is shown in Table II. Since the trigger is random, we repeat these experiments with five independently randomly generated triggers on the three datasets. In general, the ASRs are close-to-100% under all the considered models and datasets. The backdoor trigger can be easily recognized by the models, and exploited by the adversary. At the same time, the Functionality Loss is low, only around 2%. The image classification capability of the poisoned DNN models is not compromised by the backdoor trigger, hence making the trigger hard to notice. As also shown in Table II, any trigger with the same magnitude and margin can achieve nearly the same performance (including the ASR and the Functionality Loss) in the test stage.

B. Imperceptibility
Table III evaluates quantitatively and qualitatively the invisibility of the proposed trigger on the MNIST dataset. It shows that the perturbation magnitude m = 10 can provide reasonable imperceptibility. The difference between a clean image and its poisoned version is unnoticeable in the residual maps, as also corroborated quantitatively with the SSIM close to one and the LPIPS close to zero. Table IV evaluates the invisibility of the trigger on the CIFAR-10 dataset. Without loss of generality, we select at random a clean image from each of the ten classes in the CIFAR-10 dataset. We see that the difference between a clean image and its poisoned version is unnoticeable in the residual maps, and the SSIM is close to one and the LPIPS is close to zero.
The invisibility of the new trigger is also evaluated on the BTSR dataset in Table V, where a clean image and its poisoned versions under different backdoor methods are provided. With the highest SSIM and the lowest LPIPS, the proposed trigger is the least visible among all the methods. Specifically, the LPIPS of the trigger is about 5, 20, and 1.5 times lower than those of the original BadNets, Trojaned NN, and HB, respectively (see the second, fourth, sixth, and eighth rows in Table V).
We also investigate the relationship between the magnitude m and the invisibility of the proposed trigger by taking the CIFAR-10 dataset for example. As shown in Table VI, even when the ASR of the trigger is as high as 96.49% (even    The superb imperceptibility of the new trigger is the result of the large dimension and subsequently the weak per-pixel perturbation of the trigger. Specifically, the perturbation of the new trigger is dispersed throughout large areas of a poisoned image. The perturbation to each individual pixel is weak, e.g., m ≤ 10, substantially smaller than the maximum magnitude of 256 per RGB channel (i.e., less than 5%). Moreover, the uniformly randomly produced elements inside the trigger can be viewed as noises to the images poisoned. With an adequate selection of m, the trigger can be imperceptible to human eyes while remaining highly effective in attack success.

C. Attack Performance vs. Imperceptibility
Figs. 3-6 show the trade-off between the ASR and invisibility of backdoor triggers assessed on the CIFAR-10 and BTSR dataset. The invisibility is measured by LPIPS and SSIM. For a comprehensive comparison, the transparency of a trigger is adjusted by configuring a transparency parameter 0 ≤ T < 1 in BadNets and HB, as in Trojaned NN [33]. The trigger is opaque if T = 0. It is more transparent if T is larger.   Fig. 4, indicating the new trigger and BadNets can achieve both high ASRs and imperceptibility (high in SSIM and low in LPIPS). It is worth pointing out that the original design of BadNets only uses opaque triggers (i.e., T = 0). While the extended BadNets with transparent triggers can marginally outperform the new trigger, the original BadNets performs poorly in imperceptibility. Moreover, the new trigger generally has a smaller Functionality Loss than BadNets on the CIFAR-10 dataset, as revealed in Table VII.
The superiority of the new trigger to the other methods, including BadNets, is revealed on the BTSR dataset in terms of both ASR and invisibility, as shown in Figs. 5 and 6. The images in the BTSR dataset have a larger dimension (i.e., 224 × 224 pixels) than those in the CIFAR-10 dataset (i.e., 32 × 32 pixels). The larger dimension of the images allows for a larger size and smaller perturbation magnitude of the new trigger, benefiting both attack success and imperceptibility.
We also compare the new backdoor trigger with Rob-Net [34], an extension to Trojaned NN [33] by supporting multiple locations for a trigger (up to eight different locations on an image) or multiple different triggers (one per location) for a backdoored model. We adopt the setting of [34] on the trigger number and locations of RobNet: In the case of multiple locations for a trigger, we generate two poisoned versions of every clean image, each placing the trigger at a different and randomly selected location from the eight can-  didate locations. In the case of multiple triggers, we generate three poisoned versions of a clean image, each poisoned with a different trigger at a randomly selected location. The poisoning rate (i.e., the ratio of poisoned images to all input images) of RobNet is set to be no lower than the poisoning rate of the new trigger. We set the poisoning rate of the new trigger to 5%, which corresponds to 228 and 2500 poisoned images on the BTSR dataset and the CIFAR-10 dataset, respectively.
As shown in Table VIII, the new trigger achieves higher   ASRs and significantly lower Functionality Loss than RobNet in all considered scenarios. On the BTSR dataset, the trigger achieves the ASR of over 90% with at most a 3.16% loss of Functionality. In contrast, RobNet undergoes around 50% loss of Functionality, and its ASR is just about 16%. On the CIFAR-10 data, the highest ASR achieved by RobNet is 58.86% at a 8.79% Functionality Loss. By setting m to be as small as m = 3, the new trigger can achieve higher ASRs with much smaller Functionality Loss than RobNet. Moreover, the new trigger is less visible (with higher SSIM and lower LPIPS) than RobNet.
We also compare the proposed trigger with the inputagnostic adversarial trigger generation approaches, i.e., AdvGAN [22] and UAT [23]. As shown in Table IX, the new trigger achieves significantly higher ASRs than AdvGAN and UAT on all the considered datasets. Take the CIFAR-10 dataset for an example. The new trigger outperforms AdvGAN and UAT by 17.26% and 30.53%, respectively. Considering the LPIPS metric, the new trigger is the most invisible on the MNIST dataset, less visible than AdvGAN on the CIFAR-10 dataset, and less visible than UAT on the BTSR dataset. Considering the SSIM metric, the new trigger is the most invisible on both the CIFAR-10 and BSTR datasets, and less visible than UAT on the MNIST dataset.   [22] and UAT [23]. For AdvGAN, the number of epochs is 60 based on experimental tests to maximize the ASR (as no value was recommended in [22]). For UAT, the number of epochs is 10, as suggested in [23].

D. Resistance to Existing Defense Methods
One possible defense method is that a defender could decide to enumerate all possible 3D patterns (or realizations) of the trigger. A poisoned image could be potentially detected by correlating the image with every possible realization of the trigger. Nevertheless, the use of the CSPRNG ensures consistently low correlations between any two different random sequences. The trigger would only be revealed if the same trigger is picked up for correlation. A backdoored DNN model could be potentially detected by perturbing (labeled) benign images with each possible trigger realization and inputting the perturbed images into the model to gauge the misclassification rate. An image perturbed by the real trigger would be classified to a different class from its correct class, and the misclassification rate increases. The complexity of enumerating all possible trigger realizations grows exponentially with the elements in the trigger and is computationally prohibitive in practice.

1) STRIP:
In this strategy, a backdoor trigger is detected by comparing the outputs of the backdoored DNN after being fed with clean and poisoned samples. For the CIFAR-10 and MNIST datasets, we randomly select 2,000 images from a pool of 10,000 testing images and organize them into a clean group. The images in the clean group are then duplicated and poisoned with the new trigger to form a poisoned group. We superpose clean images from other classes than the selected 2,000 images, to both the clean and poisoned groups. For the BTSR dataset, we randomly select 1,000 images since there are a limited number of testing images. Fig. 7 plots the histogram of entropy. The entropy of samples with and without the new backdoor trigger has nearly the same distribution in Fig. 7(a). The entropy of both clean and poisoned CIFAR-10 samples is primarily lower than 0.2, and more than half of them are close to zero. In Figs. 7(b) and 7(c), while the distributions of the entropy of the samples with and without the backdoor trigger are different, the entropy of most samples in the BTSR and MNIST datasets ranges from 0.5 to 1.5, and from 0.5 to 1.3, respectively. It is difficult to derive a threshold to separate the clean and poisoned samples based on the entropy.
2) Spectral Signature Defense and Activation Clustering: SSD [14] and AC [15] examine the difference of internal representations (i.e., activations) between clean and poisoned images in the hidden layers (e.g., penultimate layer) of a DNN. Table X shows that SSD has a significantly lower TNR than TPR on all the three datasets, with the TNRs lower than 0.1 for all the considered values of m . This is because SSD first determines the trigger detection threshold ξ by (1), and then confirms clean samples if X − µ F , v > ξ in (2). As samples poisoned by the new trigger do not satisfy thespectrally separable condition described in Section II-B, most clean samples yield X − µ F , v < ξ. As a result, SSD classifies nearly all samples as poisoned images and obtains a bACC of around 0.5. SSD cannot detect the trigger.
Table X also shows that the bACC of AC is less than 0.7 on the different datasets, while most of its TPRs are much lower than the TNRs. The TPR and TNR of the AC method vary significantly across different parameter configurations and datasets. On the low-dimensional grey-scale MNIST images, using the Smaller method (see Section II-B) with AC allows for a TPR of 0.494 and a TNR of 0.572. The rest of the methods cause AC to misclassify all samples to be clean images and obtain a TPR of zero. On the high-dimensional   color BTSR images, the Distance method (see Section II-B) allows AC to achieve a high TNR between 0.980 and 0.989 along with a TPR ranging from 0.411 to 0.422, resulting in a small bACC value ranging from 0.696 to 0.706. Techniques that reduce the dimensions of images, such as PCA, FastICA, and t-SNE, cause little difference. AC fails to detect the new trigger, as K-Means is unsuitable for high-dimensional data while the dimension reduction of the activations can compromise the images. Other clustering methods, such as DBSCAN, Gaussian Mixture Models, and Affinity Propagation, perform worse in clustering dimensionreduced activations than K-Means [15]. Because the new trigger has a smaller perturbation magnitude than a visible trigger (e.g., the yellow square in [7]), it gets obfuscated during the dimension reduction. As a result, it is difficult to distinguish between poisoned and clean samples using the reduced activations in AC.
3) Februus: Under the assumption that the pixels corresponding to the activation hot spots in a DNN's penultimate layer are potentially a backdoor trigger, Februus removes the trigger by replacing the pixels with an image patch recovered by a pre-trained GAN model [8]. We reuse the model architecture and pre-trained GAN model provided in [8]. On the CIFAR-10 dataset, a DNN with six convolution layers and two dense (i.e., fully-connected) layers is trained with clean images and images poisoned with the new trigger. Table XI shows that Functionality and ASR are 87.8% and 92.2%, respectively. Although the ASR is reduced to 18.8% by applying Februus, the Functionality of the network is also reduced significantly to 42.5%, rendering the model useless. On the BTSR dataset, the ResNet-18 achieves the Functionality of 91.8% and the ASR of 98.7% before the trigger removal. However, the Functionality drops to 91.4% after the trigger removal, while the ASR remains barely changed. On the CIFAR-10 dataset, the significant reduction in the Functionality is due to the fact that poisoned samples with the new trigger produce similar hot spots to clean samples and Februus using CAM cannot tell their difference. As shown in Fig. 8, the hot spot (shaded in red) in a poisoned image overlaps with the object to be classified (e.g., the majority of the ship body). Furthermore, because the pixels outside the hot spot are mainly the background and provide little information about the original image, the GAN model cannot recover the original clean image patch at the hot spot. The barely changed ASR on the BTSR dataset is due to the fact that the magnitude used is too weak for CAM to produce a meaningful hot spot. Few pixels are changed by the pretrained GAN used in Februus. As a result, the trigger cannot be removed without compromising the classification capability (i.e., the Functionality) of a DNN.  Table XII shows the resistance of the new backdoor trigger to Neuron Pruning, where the threshold of terminating Neuron Pruning is over 4% reduction in Functionality Loss (i.e., the loss of classification accuracy on clean inputs), as considered in [17]. It is shown that on the BTSR dataset, Neuron Pruning can only reduce the ASR of the proposed trigger by only 3.76% (from 99.13% to 95.37%), which is even smaller than the 4.43% decrease in the classification accuracy. On the CIFAR-10 dataset, Neuron Pruning even increases the ASR by 2.55%. On the lowdimensional gray-scale MNIST dataset, the ASR drops by 22.5% for a shallow MLP model. However, the residual ASR is still as high as 74.26%, posing a considerable threat to safetyor security-critical applications.

4) Neuron Pruning:
5) Neural Cleanse: Given a potentially backdoored neural network model, Neural Cleanse obtains a set of triggers, each of which can cause misclassification to the corresponding target label and has the minimal footprint in terms of the number of occupied pixels and color intensities [18]. A backdoor is detected if any of these triggers has a significantly smaller 1 -norm than the others and has a variance larger than a  threshold, e.g., 2. The target label associated with the trigger is considered as the target label of the backdoor attack.
We examine the resistance of the new backdoor trigger to Neural Cleanse, where a backdoored ResNet-18 model is considered on the CIFAR-10 dataset. Table XIII summarizes the 1 -norm and the anomaly indices of all ten image classes, where the target label is set to be dog. The median and the median absolute fdeviation (MAD) of the 1 -norm of the triggers are 0.000549316 and 0.00033934, respectively. We see that the anomaly index values of the first two classes (i.e., airplane and car) are greater than 2, but their 1 -norm values are greater than the median (i.e., 0.000549316). Therefore, the two classes are not considered as target classes. The remaining classes (from row 4 to row 11 in Table XIII) all have their anomaly indices smaller than 2. As a result, the Neural Cleanse fails to detect the proposed backdoor trigger.
6) Spatial Smoothing: Spatial Smoothing obfuscates a trigger's perturbation by blurring the pixel values in each patch of an image [35]. A window size specifies the size of a patch on the image. The window size of 1 indicates a median filter with size of 1 × 1.
As shown in  7) Image Transformation: Defenders can destroy the perturbation caused by the backdoor trigger by performing image transformations, such as random cropping, random rotation, and horizontal flipping. Table XV demonstrates that these image transformations have no adverse effect on the ASR under the CIFAR-10 and BTSR datasets. The ASR is consistently around 98.6% under the BSTR dataset, while it increases by 0.3 to 1.5% under the CIFAR-10 dataset. The reason is twofold. Firstly, the backdoor trigger is symmetric; see Section III-B. Flipping has no impact on the performance of the trigger. Secondly, the training data contains randomly cropped and/or rotated versions of the poisoned images due to the application of data augmentation. Therefore, the DNN can still be triggered to open a backdoor by transformed images. 8) ABS: ABS is effective when the maximum reverseengineered trojan trigger's ASR (RE-ASR) of a benign model is considerably lower than the RE-ASR of its trojaned version, e.g., by 5% or more [16]. Table XVI shows that the gap is smaller than 2% on the MNIST and BTSR datasets. ABS cannot detect the backdoored MLP and ResNet-18 models trained with the new trigger. Particularly, it cannot detect suspicious neurons in the backdoored MLP model and fails to reverse-engineer any trigger. The RE-ASR is zero.
Table XVI also shows that, on the CIFAR-10 dataset, the best RE-ASR is 34.16% under the backdoored ResNet-18 model, while the maximum RE-ASR of the benign model is 24.83%. ABS can be aware of the existence of the new backdoor trigger. Nevertheless, the attacker can launch an adaptive attack to refine the backdoored model using the best reverse-engineered trigger generated by ABS. Specifically, we replace half of the clean images with images poisoned with the trigger in the poisoned training dataset, and continue to train the backdoored model for 30 epochs. The poisoned training dataset contains 5% poisoned images tampered with the proposed trigger and the attack target class label, 47.5% poisoned images tampered with the best reverse-engineered triggers and with their original class labels unchanged, and 47.5% clean images with original class labels. We see that, after the adaptive attack, the RE-ASR of the backdoored model drops to 18.26%, even lower than the maximum RE-ASR of the benign model. ABS cannot detect the backdoored models, under adaptive attacks with the new trigger.
9) Fine-tuning: Fig. 9 evaluates the robustness of the proposed trigger and the benchmarks against fine-tuning, where part of the testing data is repurposed by the defender (i.e., the recipient of the backdoored DNN model) to fine-tune the DNN model. The x-axis of the subfigures specifies the proportion of the testing data that is randomly selected to fine-tune the DNN model. The rest of the testing data is used to test the As expected, the ASRs of the considered backdoor attacks decline and the classification accuracies of the backdoored DNN models (regarding benign inputs) improve, with the increase of benign inputs used for fine-tuning the DNN models. The new trigger remains the most effective after fine-tuning, offering the highest ASR and classification accuracy on all the considered datasets; see Fig. 9. We note that BadNets provides a similar ASR and/or classification accuracy to the proposed trigger under some of the datasets; see Figs. 9(a)-9(c). But it is considered to be less effective than the new trigger due to its much worse performance in imperceptibility (see Figs. 3 and 4) and on the other datasets; see Figs. 9(d)-9(f).

E. Comparison of Run Time
Table XVII provides a quantitative comparison between the proposed approach and the existing schemes in terms of the time required to generate a trigger, denoted by T-time, and the time to produce a malicious input perturbed by the trigger, denoted by M-time. All the experiments are carried out on a server with Intel(R) Xeon(R) Gold 6258R CPU@2.70GHz and 503G memory, and an NVIDIA A100 Tensor Core GPU with 80G memory, running Python 3.7.11, Numpy 1.21.2, and PyTorch 1.10.2 installed on an Ubuntu 18.04.5 LTS operating system. The Python time module, time.time(), is called at the beginning and the end of a trigger or malicious input generation process to evaluate the T-time and M-time.
We can see from the third column of Table XVII that the trigger generation time of ours is negligible and much shorter than AdvGAN, UAT, Trojaned NN, and RobNet on all the considered datasets. We can also see from the last column of Table XVII that the malicious input generation time of our approach is the shortest among all of the considered methods under the MNIST dataset and the CIFAR-10 dataset. We note that the triggers used by BadNets and Hidden Backdoor (e.g., a color block) are selected in prior, and the trigger generation time (T-time) is not applicable.

V. CONCLUSION AND FUTURE WORK
In this paper, we proposed a new backdoor trigger, which is a uniformly randomly generated 3D binary pattern and can be horizontally and/or vertically repeated and mirrored and superposed onto three-channel images to train backdoored DNN models. While the new trigger collectively holds a strong recognizable pattern to effectively train or activate the backdoor of a DNN model, it generates weak perturbation to individual pixels and hence remains imperceptible. The complexity of the trigger generation and image perturbation is linear to the image size, and substantially lower than that of the existing triggers, such as RobNet, AdvGAN, and UAT. Extensive experiments showed that the new trigger is more than 5, 20, and 1.5 times better than the existing backdoor attacks, such as BadNets, Trojaned NN, and Hidden Backdoor, in terms of imperceptibility (LPIPS). The new trigger achieves nearly 100% ASR, and invalidates the state-of-the-art defense techniques. In the future, we will investigate countermeasures to detect and defend the new trigger.