{'fraction_non_alphanumeric': 0.03713806126731011, 'fraction_numerical': 0.01038606798153588, 'mean_word_length': 5.265775635407537, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Defending Substitution-Based Profile Pollution Attacks on Sequential Recommenders

While sequential recommender systems achieve significant improvements on capturing user dynamics, we argue that sequential recommenders are vulnerable against substitution-based profile pollution attacks. To demonstrate our hypothesis, we propose a substitution-based adversarial attack algorithm, which modifies the input sequence by selecting certain vulnerable elements and substituting them with adversarial items. In both untargeted and targeted attack scenarios, we observe significant performance deterioration using the proposed profile pollution algorithm. Motivated by such observations, we design an efficient adversarial defense method called Dirichlet neighborhood sampling. Specifically, we sample item embeddings from a convex hull constructed by multi-hop neighbors to replace the original items in input sequences. During sampling, a Dirichlet distribution is used to approximate the probability distribution in the neighborhood such that the recommender learns to combat local perturbations. Additionally, we design an adversarial training method tailored for sequential recommender systems. In particular, we represent selected items with one-hot encodings and perform gradient ascent on the encodings to search for the worst case linear combination of item embeddings in training. As such, the embedding function learns robust item representations and the trained recommender is resistant to test-time adversarial examples. Extensive experiments show the effectiveness of both our attack and defense methods, which consistently outperform baselines by a significant margin across model architectures and datasets.


INTRODUCTION
Sequential recommenders are a common approach for making personalized recommendations, such recommenders take user interaction history as input and generate potential items that may be of interest to the user [15,18,40].
Different from traditional recommenders, sequential recommendation can capture users' evolving dynamics by treating item transition patterns as temporal sequences. As such, various recent sequential recommenders (e.g., Locker [13]) consistently outperform previous state-of-the-art models.
While gradient-based methods can not be directly applied on recommenders for searching perturbations in the item space, certain heuristics can be used to perform adversarial attacks [25,34]. Based on such methods, profile pollution 1 is adopted to perform user-specific adversarial attacks [34,36]. To perform such attacks, malware can be used to access user profiles and interactions (e.g., cross-site request forgeries (CSRF) or backdoor attacks) [16,26,37].
We argue that sequential recommenders have unique vulnerabilities against substitution-based profile pollution attacks. Substitution-based adversarial attacks manipulate vulnerable items in previous or ongoing interactions to sabotage the recommendation (i.e., untargeted attacks) or manipulate the recommended items (i.e., targeted attacks) [16,20,26,37]. Previous works on profile pollution study attack algorithms and have the following limitations: (1) Existing methods designed for traditional recommenders can not be applied, or are not tailored for sequential recommenders [32,34,43]; (2) Previous methods do not explore substitution-based attacks, instead, they focus on the injection of adversarial items [36,43]; and (3) Adversarial defense methods are not discussed [24,32,34,36,43].
A similar sequential setting can be found in natural language processing (NLP) [5]. NLP adversarial attacks primarily study word-level substitutions, where a small percentage of input words are substituted [1,17,27]. To find adversarial examples in NLP, one possible method is to search among synonyms iteratively [1]. Another common approach is to select vulnerable words and replace them with adversarial ones [17,27]. Despite similarity between NLP and sequential recommendation, it is difficult to build adversarial examples for recommenders using similar techniques. The reasons are two-fold: (1) the item space is too large to enumerate without synonym information in recommender systems; (2) existing NLP attack algorithms do not exploit gradient-based methods to maximize the attack performance.
In this paper, we design a gradient-guided algorithm for substitution-based profile pollution attacks. Our attack algorithm modifies the input sequences by selecting a limited number of vulnerable elements and replacing them with carefully chosen adversarial items. By design, the substitution-based attack can perform either substitution attacks in a sequence, or inject malicious items to user interactions (by inserting and attacking an item, as in [36]). To prevent from being identified as malicious input, we impose two constraints: (1) we restrict the maximum number of substitutions in each sequence; (2) for substitute items, we additionally require similarity to the original items. Our experiment results demonstrate a significant performance deterioration in both untargeted and targeted scenarios, suggesting that the proposed substitution-based attacks pose critical threats to sequential recommenders.
Motivated by the research gap in defense methods, we additionally propose two defense methods that are tailored for sequential recommendation. Dirichlet neighborhood sampling is proposed for efficient training of robust recommenders.
Specifically, we construct multi-hop neighborhoods with the embedding matrix and sample augmented item embeddings via Dirichlet distributions to perform random substitutions. Additionally, we design an adversarial training method for sequential recommenders. In particular, we represent items with one-hot encodings and update the encodings to search for the worst case augmentation. By training the recommender on such virtual input, we extend the robustness of the recommender to a larger neighborhood. Our evaluation results show that the proposed defense methods significantly reduce performance variations under profile pollution attacks by over 50% on multiple real-world datasets. 1 In the following, we use adversarial attack and profile pollution attack interchangeably.
In this work, we focus on profile pollution and perform user-specific attacks to manipulate the recommendation results [36]. CSRF attacks manipulate user history on websites like YouTube under a black-box setting [33]. Malicious requests are exploited to bias web advertisements towards higher-paying advertisers [22]. With knowledge to the recommender, finding adversarial examples can be formulated as an optimization problem [34]. Web injections are used in unprotected websites to promote target items [43]. Yue et al. [36] propose to extract black-box recommenders and compute adversarial examples using the extracted model. Our proposed profile pollution algorithm differs from the previous methods in two aspects: (1) we extend the attack scope to substitutions in the full sequence; and (2) constraints are imposed on adversarial examples to enforce similarity between the clean and adversarial sequences.

Adversarial Attacks in Language
Given the setting of profile pollution by item substitution, a similar scenario can be found on various NLP classification tasks, where adversarial attacks are studied on word-level substitutions. NLP substitution attacks replace a small percentage of the input words with adversarial words to achieve the attack objective [1,17]. NLP adversarial examples can be generated using a genetic algorithm, where synonyms are iterated and evaluated [1]. The vulnerability of input words can be evaluated via importance scores, followed by substituting the most vulnerable words to construct adversarial examples until the model is fooled [7,17,19,27]. Despite similarity between language and sequential recommendation, it is difficult to transfer existing NLP methods to recommenders due to: (1) the lack of synonyms (i.e., neighbor items) in recommendation data; and (2) the inefficiency of existing NLP methods for large-scale or real-time attacks. Unlike previous methods, the proposed profile pollution algorithm constructs multi-hop neighborhoods and efficiently computes adversarial substitutes by exploiting gradient information from the recommender model.

Adversarial Defense Methods
Various defense methods are proposed to enhance the robustness of neural networks [9,10]. Adversarial training and smoothing methods incorporate augmented examples in training to enhance robustness against perturbations [9,38,39].
Interval bound propagation regularizes tractable upper and lower bounds to guarantee robustness [10]. Adversarial training introduced norm-bounded local perturbations to word embeddings to learn robust features [28,45]. Recently, Dirichlet neighborhood ensemble leverages synonym embeddings to build robust word representations [44]. For recommender systems, adversarial training introduces perturbations to improve model performance and generalization [3,12,35]. Multimedia recommender learns to defend perturbations on product images via adversarial training [31].
To the best of our knowledge, no previous methods attempt to enhance the robustness of recommender systems against profile pollution attacks. As such, we propose two novel defense methods tailored for sequential recommenders and show that model robustness can be significantly improved using the proposed defense methods.

Setting
3.1.1 Data. Our framework is based on sequential input, which takes user interaction history (sorted by timestamps) as input. represents input [ 1 , 2 , ..., ] of length , with each element represented in the item scope I (i.e., ∈ I).
The next item interaction +1 ∈ I after input is used as ground truth (i.e., = +1 ).

Model.
We denote the sequential recommender with function . Given input sequence , predicts a probability distribution over the item scope I. comprises of an embedding function and a sequential model , with ( ) = ( ( )). For data pair ( , ), ideally, predicts with the highest probability (i.e., = arg max ( )).

Optimization.
The learning of a sequential recommender is to maximize the probability of output item upon input . In other words, we minimize the expectation of loss L w.r.t. over data distribution X: where L represents the training loss function (i.e., ranking loss or cross entropy loss).

Adversarial Attacks and Defense
3.2.1 Assumptions. We first introduce the assumptions of our framework in the attack scenario, we formalize the problem setting with the following assumptions to define our research scope: • Data Access: Evaluation data X and item scope I are accessible to the attacker to perform the attacks.
• White-box Attacks: We assume weights of can be accessed by the attacker to compute adversarial examples.
• Limited Substitutions: In each input sequence, we impose a maximum number of substitutions to enforce the similarity between the original and adversarial examples on the sequence level. • Constraints on Item Similarity: To avoid being identified as outlier, we impose a similarity constraint with a minimum cosine similarity between the original items and the adversarial items in the embedding space.
The above assumptions are made for a direct and undisturbed evaluation of the proposed profile pollution attack algorithm (i.e., without considering external influences like model extraction). The constructed adversarial attacks are also more challenging and thus, evaluate the effectiveness of the proposed defense methods under harsh conditions. For the defense propose, we assume full access to the data and recommender models.
3.2.2 Threat Model. As described, we adopt profile pollution attacks to construct adversarial examples. We investigate untargeted attacks (i.e., demotion) and targeted attacks (i.e., promotion). We formulate both attacks below. We denote the adversarial example with ′ and formulate the attack as an optimization problem w.r.t. :

Adversarial Training.
Given the profile pollution attack framework, a robust recommender can be built by minimizing the expected loss upon perturbed input data (i.e., adversarial training). That is, we alternate between optimizing model parameters in and updating adversarial data based on to search for an equilibrium.
The adversarial examples are generated on the fly during training and being used to minimize the training loss L, such that the model is forced to adjust parameters to resist adversarial perturbations. Formally, we define the adversarial training as a minimax optimization problem w.r.t. model :

Profile Pollution Attack
The proposed algorithm can either attack history items or insert adversarial items to 'fool' recommenders. Ideally, the polluted sequences result in either performance degrade (i.e., untargeted attack) or increased exposure of the target item (i.e., targeted attack). We illustrate the proposed profile pollution attack algorithm in Figure 1, the detailed steps for both untargeted and targeted attacks are described as in Algorithm 1.
Step 1: Forward and Compute Gradients w.r.t. Input. Given input , we construct the polluted sequence ′ by substituting selected items in . First, we initialize ′ unchanged to , followed by computing the embedded sequencẽ ′ using the embedding function . We then calculate the gradients of˜′ to construct adversarial perturbations in the embedding space. In the case of targeted attack,˜′ is fed through the recommender function to compute the cross entropy loss L w.r.t. target item . Then, backward propagation is performed to retrieve the gradients ∇˜′ of˜′ (i.e. ∇˜′ = ∇˜′ L ce ( (˜′), )). For untargeted attacks, we compute the cross entropy loss w.r.t. the predicted item in the output distribution. The gradients are negated in untargeted attacks to perform gradient ascent. Compute cosine similarity : ← CosSim(˜′ − sign(∇˜′ ), ( )))∀ ∈ I; 15 Impose similarity constraint : ← 1(CosSim(˜′, ( )) ≥ )∀ ∈ I; 16 Update cosine similarity : ← ⊙ ; 17 Replace ′ in ′ : ′ ← arg max( );

end
Step 2: Select Vulnerable Items to Attack. In input sequence, items are of different vulnerability. We select vulnerable items and perform substitution on such items to achieve the best attack performance. In particular, we calculate importance scores using ∇˜′ from the previous step. Unlike existing works, we take 2 norm of the last dimension in ∇˜′ as our importance scores (i.e., ∥∇˜′ ∥), such that the vulnerability of input items can be efficiently estimated with the steepness of the gradients. We select at most items by choosing the first items from importance ranking . Based on the fast gradient sign method [9], the perturbed embeddings˜′ can be computed as follows: • Untargeted:˜′ =˜′ − sign(−∇˜′ L ( (˜′), arg max( (˜′)))) (5) • Targeted:˜′ Step 3: Project Attacks to Item Space. In the last step, we project the perturbed embeddings˜′ back to the item space. Similar to [36], we compute cosine similarity between˜′ and candidate items from I, items with higher similarity values are favored as adversarial substitutes. The idea behind it is to find closest items in the direction of the constructed adversarial embeddings using cosine similarity. We denote as the cosine similarity scores. Here, we impose another constraint to enforce item similarity, where adversarial items are required to have a minimum cosine similarity (above ) with the original items. We use the indicator function (i.e., 1) to select items fulfilling the similarity constraint.
denotes the similarity constraint results and is element-wise multiplied (i.e., ⊙) with . The final adversarial can be selected from by taking the highest similarity score. To improve the efficiency of the adversarial attack and item projection, the computation of all previous steps is performed batch-wise with matrix multiplication.

Dirichlet Neighborhood Sampling
To efficiently train a robust sequential recommender, we first propose Dirichlet neighborhood sampling, a randomized training approach to enhance model robustness. For neighborhood sampling, a connected graph is necessary to identify item neighbors. In NLP, synonym information can be used to compute multi-hop neighbors. However, there exists no natural neighbors or synonyms in recommender systems. Therefore, we propose to construct a neighborhood for each item in item scope I by computing the cosine similarity between all item pairs. We define item pair ( , ) to be neighbors if item and item have a cosine similarity greater or equal to (i.e., CosSim( , ) ≥ ). In case there exist no neighbors, items with the highest similarity values are used as 1-hop neighbors. Based on the existing 1-hop neighbors, we extend the graph to include 2-hop neighbors and construct a neighborhood dictionary to sample augmented item embeddings in training. The neighborhood dictionary is updated every epochs (we use =10 in our experiments).
For each item , we can compute a convex hull in the embedding space spanned by the item's multi-hop neighbors (i.e., vertices). The convex hull is used as the sampling space for the augmented item embedding. This allow us to sample from a Dirichlet distribution defined by the vertices in the convex hull to construct randomized embeddings.
By sampling multiple times in the neighborhood, the recommender is less sensitive to local perturbations and learns to reject adversarial items, as illustrated in Figure 2a. Formally, for item and the set of its multi-hop neighbors C (C = { ,1 , ,2 , . . . , , | C | }), we represent the augmented item with sampled from a Dirichlet distribution: where values are hyperparameters for the Dirichlet distribution and can be empirically selected for multi-hop neighbors. Using the sampled , we can compute the augmented item embeddings as follows: With increasing values, we can expect the sampled points approaching the center point of the convex hull.
During training, we randomly select and replace items in input sequence . Specifically, we select proportion of the items and sample from a Dirichlet distribution to compute augmented embeddings using the constructed neighborhoods. The virtual sequence is then used to optimize the recommender. Given the increased number of 2-hop neighbors, we select 1.0 as for 1-hop neighbors and 0.5 for 2-hop neighbors in our experiments [44].

Adversarial Training with Mixed Representations
While .
Next, we perform gradient ascent and compute the gradients w.r.t. . Specifically, the embedded sequence˜for input is first computed using the embedding function . Similar to Section 4.1, we then forward˜using the sequential model and compute the gradients based on the optimization loss to update : where is the step size. In each update, we normalize the gradients by dividing the gradients by their 2 norm. Then, step size is multiplied with the gradients before performing gradient ascent. As such, is perturbed towards the misclassification area of the sequential recommender, see Figure 2b. As is a linear combination of all items in I, we require | I | , = 1 and , ≥ 0, = 1, 2, . . . and ensure that the updated follows the same requirements within the embedding space. Therefore, we clip negative elements and rescale after each update 2 : We illustrate the proposed adversarial training with mixed representations in Figure 2b.  items in each sequence and update of the selected items three times in our experiments, we optimize the recommender with both clean input and perturbed examples to enhance the model robustness.

Model.
We select four different recommender architectures to evaluate the proposed methods, see Table 2. In particular, we adopt NARM [18], SASRec [15], BERT4Rec [30] and Locker [13] in our experiments: • Neural Attentive Recommendation Machine (NARM) is a RNN-based sequential recommender comprising of a global encoder and a local encoder. NARM utilizes an attention module to compute item representations and outputs the predictions using a similarity layer [18]. • Self-Attentive Sequential Recommendation (SASRec) leverages transformer blocks to forward items autoregressively.
Transformer blocks compute attentive representations with one-directional self-attention. SASRec consists of an embedding function, transformer blocks and an output layer [15].
• Bidirectional Encoder Representations from Transformers for Sequential Recommendation (BERT4Rec) has an similar architecture to SASRec. BERT4Rec utilizes bidirectional self-attention and adopts masked training (i.e., masked language modeling) to optimize the model [30].
• Locally Constrained Self-attentive Recommender (Locker) is similar to BERT4Rec, but proposes additional local constraints to improve self-attention. In our implementation, we use the convolution layers to model local dynamics. Locker is also optimized via masked training [13].

Evaluation.
We follow [15,30,36] to perform evaluation. We adopt leave-one-out method and use the last two items in each sequence for validation and testing. We adopt normalized discounted cumulative gain NDCG@10 and Recall@10 as evaluation metrics. In the attack experiments, 15 items are selected as target items across different popularity groups for targeted attacks, we evaluate on the polluted profiles and compute the average results.

Implementation.
We follow the original papers to implement sequential recommenders [13,15,18,30]. Unless explicitly mentioned, hyperparameters are from the original works. All models are trained without warmup using Adam optimizer with learning rate of 0.001, weight decay 0.01 and batch size of 64. Similar to [15,30,36], we set the maximum sequence lengths of ML-1M to be 200 and 50 for other datasets. For profile pollution, we use = 2 as the maximum substitution for ML-1M and and = 1 for other datasets. The minimum cosine similarity of = 0.5 is imposed for both attack and defense methods. In our defense experiments, we set substitution probability to be 0. 5

RQ1: Can We Attack Recommenders using Substitution-based Profile Pollution?
We first evaluate the attack performance of the proposed profile pollution algorithm on all model architectures and datasets, the results are presented in Table 3. Since we restrain from substituting more than two items, we adopt SimAlter as the baseline method, which utilizes neighbor items as adversarial items to perform attacks [36]. The table  Table 4. Evaluation results of the defense methods under profile pollution attacks. Untargeted attack results are denoted with U (higher the better), targeted attacks are denoted with T (lower the better). Before denotes evaluation results on the clean model, while Dirichlet and AdvTrain represent the adopted defense methods.
From the results we observe the following: (1) By comparing clean performance (i.e., Before) and metric numbers under attack (i.e., SimAlter and Ours), recommender models demonstrate vulnerability against profile pollution attacks.
For example, recommenders suffer from over 50% performance deterioration in untargeted attacks on ML-1M. performance drop for untargeted attacks on LastFM dataset, while the same setting yields 37.5% deterioration on Beauty.
Overall, the results suggest that the proposed algorithm can effectively perform profile pollution attacks.

RQ2: How the Proposed Defense Methods Perform under Profile Pollution Attacks?
We study the effectiveness of defense methods by evaluating profile pollution attacks on recommenders trained with the defense methods. In particular, we adopt identical training conditions and apply the proposed methods (i.e., Dirichlet neighborhood sampling and adversarial training  Table 6. Influence of recommender architecture on profile pollution attacks. We additionally study the performance variation w.r.t. item popularity. Specifically, we study targeted attacks on sequential recommenders and select target items of different popularity. Target items are divided into three subgroups upon their popularity (i.e., number of total occurrences) in the dataset. Popular items denote the 1 3 most popular items within the item scope. Middle and Bottom items refer to the next 1 3 and the last 1 3 of the items in I. We present the item distribution of all datasets in Figure 3, we present profile pollution results of the three popularity subgroups averaged across all recommender models. Similarly, performance variations are report with Δ of NDCG@10 and Recall@10 in suggest that popular items exhibit the highest vulnerability against profile pollution attacks, a potential reason for such vulnerability is the higher number of neighbors, which increases the number of possible adversarial substitutes.

RQ3: How Recommender Performance and Robustness Change with Different Model Architectures?
We study the vulnerability of different recommender models and performance of the proposed defense methods.
In particular, we study the attacks on all datasets and compare the original performance and performance changes of different recommender architectures under profile pollution attacks. We present the averaged results of each recommender architecture in Table 6 for a direct comparison. The visualized results are presented in Figure 4a and Here are our observations: (1) We observe that adversarial training has limited influence on recommendation performance. For example, our results on transformer-based models demonstrate slight performance changes, possibly due to the improved item representations. (2) By comparing performance variations before and after profile pollution attacks in Table 6, we observe that NARM is more vulnerable and demonstrates higher performance variations both before and after applying the defense methods.  blocks to Locker, SASRec and BERT4Rec are more vulnerable and suffer higher performance deterioration even when adversarially trained. For instance, targeted attacks on popular items increase NDCG@10 variation by 31.3% on SASRec and 27.2% on BERT4Rec, compared to only 24.8% of Locker. Overall, transformer-based sequential recommenders (i.e., SASRec, BERT4Rec and Locker) demonstrate improved robustness of similar magnitude compared to RNN-based model (i.e., NARM). Comparable trends can be observed when we apply adversarial training on sequential recommenders.
The reason for such robustness may be traced back to the introduction of global attention in transformer models.
Additionally, BERT4Rec and Locker utilize masked training to improve the capture of global context. In contrast, NARM relies on local temporal patterns and can be more sensitive to substitution-based perturbations.

CONCLUSION AND FUTURE WORK
In this work, we study the effectiveness of substitution-based profile pollution attacks on sequential recommenders.
We design a profile pollution attack algorithm and evaluate on multiple datasets and state-of-the-art recommenders, where the proposed method successfully poses threats to sequential recommenders for both the untargeted and targeted attack scenarios. Then, we explore the defense methods by sampling augmented item representations from a Dirichlet distribution within multi-hop neighborhoods. We additionally design an adversarial training method to search for the worst-case augmentation and enhance model robustness. To the best of our knowledge, this is the first work on defending substitution-based profile pollution attacks for sequential recommenders. Our experimental results suggest that by applying the proposed defense methods, sequential recommender systems can learn robust item representations and demonstrate significantly reduced performance variations under profile pollution attacks.
The proposed methods have certain limitations. For example, our attack and defense methods are designed for sequential recommenders and might not be applicable for other recommender systems. Additionally, the profile pollution experiments are performed under a white-box assumption, rendering the proposed method less accessible in real-world scenarios. Despite having introduced Dirichlet neighborhood sampling, we have not discussed different choices of values for multi-hop neighbors to exploit the potential benefits of the Dirichlet distribution. For future work, we plan to relax our assumptions and explore certified defense methods for sequential recommenders. For example, it would be interesting to explore interval bound propagation to provide guaranteed defense. Additionally, we can design model-agnostic adversarial attack and defense algorithms to extend our framework to other architectures.