{'fraction_non_alphanumeric': 0.0391202212707915, 'fraction_numerical': 0.008771265108172622, 'mean_word_length': 4.835396860490825, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Recommender Transformers with Behavior Pathways

Sequential recommendation requires the recommender to capture the evolving behavior characteristics from logged user behavior data for accurate recommendations. However, user behavior sequences are viewed as a script with multiple ongoing threads intertwined. We find that only a small set of pivotal behaviors can be evolved into the user's future action. As a result, the future behavior of the user is hard to predict. We conclude this characteristic for sequential behaviors of each user as the Behavior Pathway. Different users have their unique behavior pathways. Among existing sequential models, transformers have shown great capacity in capturing global-dependent characteristics. However, these models mainly provide a dense distribution over all previous behaviors using the self-attention mechanism, making the final predictions overwhelmed by the trivial behaviors not adjusted to each user. In this paper, we build the Recommender Transformer (RETR) with a novel Pathway Attention mechanism. RETR can dynamically plan the behavior pathway specified for each user, and sparingly activate the network through this behavior pathway to effectively capture evolving patterns useful for recommendation. The key design is a learned binary route to prevent the behavior pathway from being overwhelmed by trivial behaviors. We empirically verify the effectiveness of RETR on seven real-world datasets and RETR yields state-of-the-art performance.


Introduction
Recommender systems [14,21,39] have been widely adopted in real-world industrial applications such as E-commerce and social media. Benefiting from the increase in computing power and model capacity, some recent efforts formulate recommendation as a time-series forecasting problem, known as sequential recommendation [16,28,6]. The core idea of this field is to infer upcoming actions based on user's historical behaviors, which are reorganized as time-ordered sequences. This intuitive modeling of recommendation is proved time-sensitive and context-aware to make precise predictions. be viewed as a script with multiple ongoing threads intertwined. And only key clues suggest what will happen next. In sequential recommendation, we find that only a small part of pivotal behaviors can be evolved into the use's future action. And we conclude this characteristics of sequential behaviors as the Behavior Pathway.
Different users have their unique behavior pathways and they can be grouped into three categories: • Correlated Behavior Pathway: A user's behavior pathway is closely associated with behaviors at a certain period. As shown in the second line of Figure 1, the mouse is clicked many times recently, leading to the final decision to buy a mouse. • Casual Behavior Pathway: A user's behavior pathway is interested in a specific item at casual times. As shown in the first line of Figure 1, the backpack is randomly clicked sequentially. • Drifted Behavior Pathway: A user's behavior pathway in a particular brand might drift over time. As shown in the third line of Figure 1, the user was initially interested in a keyboard, but suddenly became interested in buying a phone at last.
It's challenging to capture these aspects dynamically for each user to make precise recommendations.
Motivated by the Pathways [8], a new way of thinking about AI, which builds a single model that is sparsely activated for all tasks with small pathways through the network called into action as needed, we propose a novel Recommender Transformer (RETR) with a Pathway Attention mechanism. RETR dynamically explores behavior pathways for different users and then captures evolving patterns through these pathways effectively. To be specific, the user-dependent pathway attention, which incorporates a pathway router, determines whether or not a behavior token will be maintained in the behavior pathway. Technically, the pathway router generates a customized binary route for each token based on their information redundancy. Recommender Transformers are stacked, and successive pathway routers constitute a hierarchical evolution pathway of user behaviors. To enable the pathway router modules to be end-to-end optimized, we adopt the Gumbel-Softmax [15] sampling strategy to overcome the non-differentiable problem of sampling from a Bernoulli distribution.
To effectively capture the evolving patterns via the behavior pathway, our pathway attention mechanism makes our RETR mainly attend to the obtained pathway. We force the model to focus on the most informative behaviors by using the query routed through the behavior pathway. We cut off the interaction from the off-pathway behaviors of the query. Compared with using all previous behaviors, our pathway attention mechanism is obviously more effective and can avoid the most informative tokens being overwhelmed by trivial behaviors. To validate the effectiveness of our approach, we conduct experiments on seven real-world competitive datasets for sequential recommendations and RETR achieves state-of-the-art performance. Our main contributions can be summarized as follows: • Our work is the first to propose the concept of behavior pathway for sequential recommendation. We find the key to the recommender is to dynamically capture the behavior pathway for each user.
• We propose the novel recommender transformer (RETR) with a novel pathway attention mechanism, which can generate the behavior pathway hierarchically and capture the evolving patterns dynamically through the pathway. • We validate the effectiveness of RETR on seven real-world datasets of different scales across different scenarios for sequential recommendations and achieve state-of-the-art performance.

Related Work
Traditional recommendation approaches. Capturing evolving behavior characteristics is crucial for many online applications, such as advertising, social media and E-commerce, and it is the key challenge for sequential recommendation [1,16,7,36,9,24,5,41,20]. Traditional recommendation approach, such as the collaborative filtering (CF) [13] based on matrix approximation [18,19], always assumes that the user's behavior is static. However, in practice, user behaviors often change over time due to various reasons, making the CF deteriorate in a real-world application.
Sequential recommendation approaches. To overcome this challenge, some methods, such as FPMC [12] and HRM [30], use Markov chains to capture sequential patterns by learning userspecific transition matrices. Higher-order Markov Chains assume the next action is related to several previous actions. Benefit from this strong inductive bias, MC-based methods [12,11] show superior performance in capturing short-term patterns. At the same time, there is a potential state space explosion problem when these approaches are faced with different possible sequences [31]. In recent years, many works have been using the deep neural network for sequential recommendation. For example, the GRURec [14] and the RepeatNet [25] adopt the recurrent network to capture dynamic patterns from the user behaviors dependent on sequence positions. The RNN-based models achieve competitive performance in capturing short-term behavior patterns but cannot capture long-term behavior patterns effectively. The CNN-based model, such as Caser [29], applies convolutional operations to extract transitions while tending to overlook the intrinsic relationship across user behaviors. The GNN-based methods, such as SRGNN [33] and GCSAN [34], model behavior sequences as graph-structured data and incorporate an attention mechanism for a session-based recommendation. In addition, DIN [40] uses the gate mechanism to weight different user behaviors. However, concatenating all behaviors makes these models overlook the sequential characteristics.
Transformer-based models for Sequential Recommendation. The transformer-based models have the strong capacity to capture behavior patterns via the attention mechanism, achieving state-of-theart performance while involving many parameters. SASRec [16], BertRec [28] and SSE-PT [32] introduce the transformer architecture into sequential recommendation, which might lead to the over-parameterized architecture of Transformer-based methods. These models capture the evolving patterns by the self-attention mechanism, interacting with all previous behaviors. However, dense interactions will make the model not adapt to different users and overwhelm behavior pathways. To tackle this challenge, our paper builds the Recommender Transformer (RETR) with a new Pathway Attention mechanism that is dynamically activated for the behavior pathway of all users. Distinct from the previous routing architecture like Switch Transformer [10] using the MoE [27] structure for natural language tasks, our RETR is designed explicitly for sequential recommendation. Our RETR uses the pathway router to adaptively route the sequential behavior of each user rather than routing the experts of feed-forward networks in switch transformer.

Method
Suppose that we have a set of users and items, denoted by U and I respectively. In the task of sequential recommendation, chronologically-ordered behaviors of a user u ∈ U could be represented by a user-interacted item sequence: {i 1 , · · · , i n }. Formally, given a user u with her or his behavior sequence {i 1 , · · · , i n }, the goal of sequential recommendation is to predict the next item the user u would interact with at the (n + 1)-th step, denoted as p (i n+1 | i 1:n ).
As aforementioned, we highlight the key to sequential recommendation as the exploration of usertailored behavior pathways, through which evolving characteristics could be learned. Motivated by this, we propose a novel Recommender Transformer (RETR) with a new Pathway Attention, the core subassembly of which is a pathway router. Besides the modification of architecture, we additionally introduce a hierarchical update strategy for the behavior pathway in the feed-forward procedure.

Pathway Attention
Gumbel-Softmax llustration of our proposed approach. AdaShare learns the layer sharing pattern among multiple h predicting a select-or-skip policy decision sampled from the learned task-specific policy distribution ese select-or-skip vectors define which blocks should be executed in different tasks. A block is said across two tasks if it is being used by both of them or task-specific if it is being used by only one dicting the output. During training, both policy logits and network parameters are jointly learned ard back-propagation through Gumbel-Softmax Sampling. We use task-specific losses and policy ons (to encourage sparsity and sharing) in training. Best viewed in color. ask for predicting the output. In this way, the select-or-skip policy of all blocks and tasks k}lL,kK ) determines the adaptive feature sharing mechanism over the given task set T . mber of potential configurations for U is 2 L⇥K which grows exponentially with the blocks and tasks, it becomes intractable to manually find such a U to get the optimal ring pattern in multi-task learning. Instead of handcrafting this policy, we adopt Gumbelampling [25] to optimize U jointly with the network parameters W through standard gation. Moreover, we introduce two policy regularizations to achieve effective knowledge a compact multi-task network, as well as a curriculum learning strategy to stabilize the on in the early stages. After the training finishes, we sample the binary decision ul,k for l from ul,k to decide what blocks to select or skip in the task Tk. Specifically, with the select-or-skip decisions, we form a novel and non-trivial network architecture for MTL sharing, and share knowledge at different levels across all tasks in a flexible and efficient st time, when a novel input is presented to the multi-task network, the optimal policy is electively choosing what blocks to compute for each task. Our proposed approach not only s positive sharing among tasks via shared blocks but also minimizes negative interference sk-specific blocks when necessary.
a Task-Specific Policy. In AdaShare, we learn the select-or-skip policy U and network jointly through standard back-propagation from our designed loss functions. However, t-or-skip policy ul,k is discrete and non-differentiable and this makes direct optimization herefore, we adopt Gumbel-Softmax Sampling [25] to resolve this non-differentiability direct optimization of the discrete policy ul,k using back-propagation.
oftmax Sampling. The Gumbel-Softmax trick [25,36] is a simple and effective way to the original non-differentiable sample from a discrete distribution with a differentiable m a corresponding Gumbel-Softmax distribution. We let ⇡l,k = [1 ↵l,k, ↵l,k] be the n vector of the binary random variable ul,k that we want to optimize, where the logit ↵l,k the probability that the l-th block is selected to execute in the task Tk.
-Softmax Sampling, instead of directly sampling a select-or-skip decision ul,k for the l-th e task Tk from its distribution ⇡l,k, we generate it as, ul,k = arg max j2{0,1} log ⇡l,k(j) + Gl,k(j) , = log( log Ul,k) is a standard Gumbel distribution with Ul,k sampled from a uniform bution Unif(0, 1). To remove the non-differentiable argmax operation in Eq. 1, the Gumbel rick relaxes one-hot(ul,k) 2 {0, 1} 2 (the one-hot encoding of ul,k) to vl,k 2 R 2 (the soft kip decision for the l-th block in Tk) with the reparameterization trick [25]: vl,k(j) = exp (log ⇡l,k(j) + Gl,k(j))/⌧ P i2{0,1} exp (log ⇡l,k(i) + Gl,k(i))/⌧ , 4 Gumbel-Softmax 2: Illustration of our proposed approach. AdaShare learns the layer sharing pattern among multiple rough predicting a select-or-skip policy decision sampled from the learned task-specific policy distribution These select-or-skip vectors define which blocks should be executed in different tasks. A block is said ared across two tasks if it is being used by both of them or task-specific if it is being used by only one predicting the output. During training, both policy logits and network parameters are jointly learned andard back-propagation through Gumbel-Softmax Sampling. We use task-specific losses and policy zations (to encourage sparsity and sharing) in training. Best viewed in color.
e task for predicting the output. In this way, the select-or-skip policy of all blocks and tasks ul,k}lL,kK) determines the adaptive feature sharing mechanism over the given task set T .
number of potential configurations for U is 2 L⇥K which grows exponentially with the r of blocks and tasks, it becomes intractable to manually find such a U to get the optimal sharing pattern in multi-task learning. Instead of handcrafting this policy, we adopt Gumbelx Sampling [25] to optimize U jointly with the network parameters W through standard ropagation. Moreover, we introduce two policy regularizations to achieve effective knowledge in a compact multi-task network, as well as a curriculum learning strategy to stabilize the zation in the early stages. After the training finishes, we sample the binary decision ul,k for lock l from ul,k to decide what blocks to select or skip in the task Tk. Specifically, with the the select-or-skip decisions, we form a novel and non-trivial network architecture for MTL ter-sharing, and share knowledge at different levels across all tasks in a flexible and efficient t test time, when a novel input is presented to the multi-task network, the optimal policy is d, selectively choosing what blocks to compute for each task. Our proposed approach not only ages positive sharing among tasks via shared blocks but also minimizes negative interference g task-specific blocks when necessary.
ing a Task-Specific Policy. In AdaShare, we learn the select-or-skip policy U and network s W jointly through standard back-propagation from our designed loss functions. However, lect-or-skip policy ul,k is discrete and non-differentiable and this makes direct optimization lt. Therefore, we adopt Gumbel-Softmax Sampling [25] to resolve this non-differentiability able direct optimization of the discrete policy ul,k using back-propagation.
el-Softmax Sampling. The Gumbel-Softmax trick [25,36] is a simple and effective way to utes the original non-differentiable sample from a discrete distribution with a differentiable from a corresponding Gumbel-Softmax distribution. We let ⇡l,k = [1 ↵l,k, ↵l,k] be the tion vector of the binary random variable ul,k that we want to optimize, where the logit ↵l,k nts the probability that the l-th block is selected to execute in the task Tk.
bel-Softmax Sampling, instead of directly sampling a select-or-skip decision ul,k for the l-th n the task Tk from its distribution ⇡l,k, we generate it as, Gl,k = log( log Ul,k) is a standard Gumbel distribution with Ul,k sampled from a uniform stribution Unif(0, 1). To remove the non-differentiable argmax operation in Eq. 1, the Gumbel x trick relaxes one-hot(ul,k) 2 {0, 1} 2 (the one-hot encoding of ul,k) to vl,k 2 R 2 (the soft or-skip decision for the l-th block in Tk) with the reparameterization trick [25]: 4 Gumbel-Softmax ure 2: Illustration of our proposed approach. AdaShare learns the layer sharing pattern among multiple s through predicting a select-or-skip policy decision sampled from the learned task-specific policy distribution its). These select-or-skip vectors define which blocks should be executed in different tasks. A block is said e shared across two tasks if it is being used by both of them or task-specific if it is being used by only one for predicting the output. During training, both policy logits and network parameters are jointly learned g standard back-propagation through Gumbel-Softmax Sampling. We use task-specific losses and policy larizations (to encourage sparsity and sharing) in training. Best viewed in color.
y one task for predicting the output. In this way, the select-or-skip policy of all blocks and tasks = {ul,k}lL,kK) determines the adaptive feature sharing mechanism over the given task set T .
the number of potential configurations for U is 2 L⇥K which grows exponentially with the ber of blocks and tasks, it becomes intractable to manually find such a U to get the optimal ture sharing pattern in multi-task learning. Instead of handcrafting this policy, we adopt Gumbeltmax Sampling [25] to optimize U jointly with the network parameters W through standard k-propagation. Moreover, we introduce two policy regularizations to achieve effective knowledge ring in a compact multi-task network, as well as a curriculum learning strategy to stabilize the imization in the early stages. After the training finishes, we sample the binary decision ul,k for h block l from ul,k to decide what blocks to select or skip in the task Tk. Specifically, with the p of the select-or-skip decisions, we form a novel and non-trivial network architecture for MTL ameter-sharing, and share knowledge at different levels across all tasks in a flexible and efficient . At test time, when a novel input is presented to the multi-task network, the optimal policy is owed, selectively choosing what blocks to compute for each task. Our proposed approach not only ourages positive sharing among tasks via shared blocks but also minimizes negative interference using task-specific blocks when necessary.
rning a Task-Specific Policy. In AdaShare, we learn the select-or-skip policy U and network ghts W jointly through standard back-propagation from our designed loss functions. However, h select-or-skip policy ul,k is discrete and non-differentiable and this makes direct optimization cult. Therefore, we adopt Gumbel-Softmax Sampling [25] to resolve this non-differentiability enable direct optimization of the discrete policy ul,k using back-propagation.
mbel-Softmax Sampling. The Gumbel-Softmax trick [25,36] is a simple and effective way to stitutes the original non-differentiable sample from a discrete distribution with a differentiable ple from a corresponding Gumbel-Softmax distribution. We let ⇡l,k = [1 ↵l,k, ↵l,k] be the tribution vector of the binary random variable ul,k that we want to optimize, where the logit ↵l,k resents the probability that the l-th block is selected to execute in the task Tk.
umbel-Softmax Sampling, instead of directly sampling a select-or-skip decision ul,k for the l-th ck in the task Tk from its distribution ⇡l,k, we generate it as, re Gl,k = log( log Ul,k) is a standard Gumbel distribution with Ul,k sampled from a uniform . distribution Unif(0, 1). To remove the non-differentiable argmax operation in Eq. 1, the Gumbel tmax trick relaxes one-hot(ul,k) 2 {0, 1} 2 (the one-hot encoding of ul,k) to vl,k 2 R 2 (the soft ct-or-skip decision for the l-th block in Tk) with the reparameterization trick [25]: 4 Gumbel-Softmax Figure 2: Illustration of our proposed approach. AdaShare learns the layer sharing pattern among multiple tasks through predicting a select-or-skip policy decision sampled from the learned task-specific policy distribution (logits). These select-or-skip vectors define which blocks should be executed in different tasks. A block is said to be shared across two tasks if it is being used by both of them or task-specific if it is being used by only one task for predicting the output. During training, both policy logits and network parameters are jointly learned using standard back-propagation through Gumbel-Softmax Sampling. We use task-specific losses and policy regularizations (to encourage sparsity and sharing) in training. Best viewed in color.
only one task for predicting the output. In this way, the select-or-skip policy of all blocks and tasks (U = {ul,k}lL,kK) determines the adaptive feature sharing mechanism over the given task set T .
As the number of potential configurations for U is 2 L⇥K which grows exponentially with the number of blocks and tasks, it becomes intractable to manually find such a U to get the optimal feature sharing pattern in multi-task learning. Instead of handcrafting this policy, we adopt Gumbel-Softmax Sampling [25] to optimize U jointly with the network parameters W through standard back-propagation. Moreover, we introduce two policy regularizations to achieve effective knowledge sharing in a compact multi-task network, as well as a curriculum learning strategy to stabilize the optimization in the early stages. After the training finishes, we sample the binary decision ul,k for each block l from ul,k to decide what blocks to select or skip in the task Tk. Specifically, with the help of the select-or-skip decisions, we form a novel and non-trivial network architecture for MTL parameter-sharing, and share knowledge at different levels across all tasks in a flexible and efficient way. At test time, when a novel input is presented to the multi-task network, the optimal policy is followed, selectively choosing what blocks to compute for each task. Our proposed approach not only encourages positive sharing among tasks via shared blocks but also minimizes negative interference by using task-specific blocks when necessary.
Learning a Task-Specific Policy. In AdaShare, we learn the select-or-skip policy U and network weights W jointly through standard back-propagation from our designed loss functions. However, each select-or-skip policy ul,k is discrete and non-differentiable and this makes direct optimization difficult. Therefore, we adopt Gumbel-Softmax Sampling [25] to resolve this non-differentiability and enable direct optimization of the discrete policy ul,k using back-propagation.
Gumbel-Softmax Sampling. The Gumbel-Softmax trick [25,36] is a simple and effective way to substitutes the original non-differentiable sample from a discrete distribution with a differentiable sample from a corresponding Gumbel-Softmax distribution. We let ⇡l,k = [1 ↵l,k, ↵l,k] be the distribution vector of the binary random variable ul,k that we want to optimize, where the logit ↵l,k represents the probability that the l-th block is selected to execute in the task Tk.
In Gumbel-Softmax Sampling, instead of directly sampling a select-or-skip decision ul,k for the l-th block in the task Tk from its distribution ⇡l,k, we generate it as, where Gl,k = log( log Ul,k) is a standard Gumbel distribution with Ul,k sampled from a uniform i.i.d. distribution Unif(0, 1). To remove the non-differentiable argmax operation in Eq. 1, the Gumbel Softmax trick relaxes one-hot(ul,k) 2 {0, 1} 2 (the one-hot encoding of ul,k) to vl,k 2 R 2 (the soft select-or-skip decision for the l-th block in Tk) with the reparameterization trick [25]: 4 Gumbel-Softmax Figure 2: Illustration of our proposed approach. AdaShare learns the layer sharing pattern among multiple tasks through predicting a select-or-skip policy decision sampled from the learned task-specific policy distribution (logits). These select-or-skip vectors define which blocks should be executed in different tasks. A block is said to be shared across two tasks if it is being used by both of them or task-specific if it is being used by only one task for predicting the output. During training, both policy logits and network parameters are jointly learned using standard back-propagation through Gumbel-Softmax Sampling. We use task-specific losses and policy regularizations (to encourage sparsity and sharing) in training. Best viewed in color.
only one task for predicting the output. In this way, the select-or-skip policy of all blocks and tasks (U = {ul,k}lL,kK) determines the adaptive feature sharing mechanism over the given task set T .
As the number of potential configurations for U is 2 L⇥K which grows exponentially with the number of blocks and tasks, it becomes intractable to manually find such a U to get the optimal feature sharing pattern in multi-task learning. Instead of handcrafting this policy, we adopt Gumbel-Softmax Sampling [25] to optimize U jointly with the network parameters W through standard back-propagation. Moreover, we introduce two policy regularizations to achieve effective knowledge sharing in a compact multi-task network, as well as a curriculum learning strategy to stabilize the optimization in the early stages. After the training finishes, we sample the binary decision ul,k for each block l from ul,k to decide what blocks to select or skip in the task Tk. Specifically, with the help of the select-or-skip decisions, we form a novel and non-trivial network architecture for MTL parameter-sharing, and share knowledge at different levels across all tasks in a flexible and efficient way. At test time, when a novel input is presented to the multi-task network, the optimal policy is followed, selectively choosing what blocks to compute for each task. Our proposed approach not only encourages positive sharing among tasks via shared blocks but also minimizes negative interference by using task-specific blocks when necessary.
Learning a Task-Specific Policy. In AdaShare, we learn the select-or-skip policy U and network weights W jointly through standard back-propagation from our designed loss functions. However, each select-or-skip policy ul,k is discrete and non-differentiable and this makes direct optimization difficult. Therefore, we adopt Gumbel-Softmax Sampling [25] to resolve this non-differentiability and enable direct optimization of the discrete policy ul,k using back-propagation.
Gumbel-Softmax Sampling. The Gumbel-Softmax trick [25,36] is a simple and effective way to substitutes the original non-differentiable sample from a discrete distribution with a differentiable sample from a corresponding Gumbel-Softmax distribution. We let ⇡l,k = [1 ↵l,k, ↵l,k] be the distribution vector of the binary random variable ul,k that we want to optimize, where the logit ↵l,k represents the probability that the l-th block is selected to execute in the task Tk.
In Gumbel-Softmax Sampling, instead of directly sampling a select-or-skip decision ul,k for the l-th block in the task Tk from its distribution ⇡l,k, we generate it as, where Gl,k = log( log Ul,k) is a standard Gumbel distribution with Ul,k sampled from a uniform i.i.d. distribution Unif(0, 1). To remove the non-differentiable argmax operation in Eq. 1, the Gumbel Softmax trick relaxes one-hot(ul,k) 2 {0, 1} 2 (the one-hot encoding of ul,k) to vl,k 2 R 2 (the soft select-or-skip decision for the l-th block in Tk) with the reparameterization trick [25]: vl,k(j) = exp (log ⇡l,k(j) + Gl,k(j))/⌧ P i2{0,1} exp (log ⇡l,k(i) + Gl,k(i))/⌧ , 4 Gumbel-Softmax Figure 2: Illustration of our proposed approach. AdaShare learns the layer sharing pattern among multiple tasks through predicting a select-or-skip policy decision sampled from the learned task-specific policy distribution (logits). These select-or-skip vectors define which blocks should be executed in different tasks. A block is said to be shared across two tasks if it is being used by both of them or task-specific if it is being used by only one task for predicting the output. During training, both policy logits and network parameters are jointly learned using standard back-propagation through Gumbel-Softmax Sampling. We use task-specific losses and policy regularizations (to encourage sparsity and sharing) in training. Best viewed in color.
only one task for predicting the output. In this way, the select-or-skip policy of all blocks and tasks (U = {ul,k}lL,kK) determines the adaptive feature sharing mechanism over the given task set T .
As the number of potential configurations for U is 2 L⇥K which grows exponentially with the number of blocks and tasks, it becomes intractable to manually find such a U to get the optimal feature sharing pattern in multi-task learning. Instead of handcrafting this policy, we adopt Gumbel-Softmax Sampling [25] to optimize U jointly with the network parameters W through standard back-propagation. Moreover, we introduce two policy regularizations to achieve effective knowledge sharing in a compact multi-task network, as well as a curriculum learning strategy to stabilize the optimization in the early stages. After the training finishes, we sample the binary decision ul,k for each block l from ul,k to decide what blocks to select or skip in the task Tk. Specifically, with the help of the select-or-skip decisions, we form a novel and non-trivial network architecture for MTL parameter-sharing, and share knowledge at different levels across all tasks in a flexible and efficient way. At test time, when a novel input is presented to the multi-task network, the optimal policy is followed, selectively choosing what blocks to compute for each task. Our proposed approach not only encourages positive sharing among tasks via shared blocks but also minimizes negative interference by using task-specific blocks when necessary.
Learning a Task-Specific Policy. In AdaShare, we learn the select-or-skip policy U and network weights W jointly through standard back-propagation from our designed loss functions. However, each select-or-skip policy ul,k is discrete and non-differentiable and this makes direct optimization difficult. Therefore, we adopt Gumbel-Softmax Sampling [25] to resolve this non-differentiability and enable direct optimization of the discrete policy ul,k using back-propagation.
Gumbel-Softmax Sampling. The Gumbel-Softmax trick [25,36] is a simple and effective way to substitutes the original non-differentiable sample from a discrete distribution with a differentiable sample from a corresponding Gumbel-Softmax distribution. We let ⇡l,k = [1 ↵l,k, ↵l,k] be the distribution vector of the binary random variable ul,k that we want to optimize, where the logit ↵l,k represents the probability that the l-th block is selected to execute in the task Tk.

Multi-head Attention MatMul MatMul
Gumbel-Softmax rns the layer sharing pattern among multiple m the learned task-specific policy distribution e executed in different tasks. A block is said r task-specific if it is being used by only one and network parameters are jointly learned pling. We use task-specific losses and policy st viewed in color. ect-or-skip policy of all blocks and tasks ing mechanism over the given task set T .
K which grows exponentially with the nually find such a U to get the optimal ndcrafting this policy, we adopt Gumbeltwork parameters W through standard arizations to achieve effective knowledge iculum learning strategy to stabilize the , we sample the binary decision u l,k for kip in the task T k . Specifically, with the on-trivial network architecture for MTL across all tasks in a flexible and efficient ulti-task network, the optimal policy is ch task. Our proposed approach not only but also minimizes negative interference he select-or-skip policy U and network our designed loss functions. However, iable and this makes direct optimization [25] to resolve this non-differentiability ing back-propagation. [25,36] is a simple and effective way to iscrete distribution with a differentiable n. We let ⇡ l,k = [1 ↵ l,k , ↵ l,k ] be the e want to optimize, where the logit ↵ l,k xecute in the task T k . a select-or-skip decision u l,k for the l-th it as,

Recommender Transformer
Considering the limitation of Transformers [4] for sequential recommendation, we renovate the vanilla architecture to the Recommender Transformer ( Figure 2) with a Pathway Attention mechanism.
Model inputs. To obtain the model inputs, we follow the sliding window practice and transform the user's behavior sequence into a fixed-length-N sequence s = (s 1 , s 2 , . . . , s N ). Then we produce an item embedding matrix E I ∈ R |I|×d , where d is the embedding dimensionality. We perform a look-up operation from E I to retrieve the input embedding matrix E s ∈ R N ×d for sequence s. Besides, we also add a learnable position embedding P s ∈ R N ×d for sequence s. Finally, we can generate the input embedding of each behavior sequence s as X s = E s + P s ∈ R N ×d .
Overall architecture. Recommender Transformer is characterized by stacking the Pathway Attention blocks and feed-forward layers alternately, containing L blocks. This stacking structure is conducive to learning behavior representations hierarchically. The overall equations of block l are formalized as: where Z l ∈ R N ×d , l ∈ {1, · · · , L} denotes the output of the l-th block. The initial input Z 0 = X s ∈ R N ×d represents the raw behavior embedding. R l−1 is the previous route from the block l − 1 and we initialize all elements in the route R 0 to 1. Path-MSA(·) is to conduct the pathway attention. LN(·) is to conduct layer normalization [3] and FFN represents the point-wise feed-forward network [4].

Pathway Attention
Note that the single-branch self-attention mechanism [4] in vanilla transformer cannot model the behavior pathway dynamically, resulting in key behaviors being overwhelmed by these non-pivotal ones. To solve this problem, we propose the Pathway Attention mechanism, as shown in Figure 2, which can dynamically attend to the behavior pathway of pivotal behavior tokens.
Pathway router. The pathway attention employs a sequence-adaptive pathway router to custom-tailor behavior pathway routes for users. The router generates a binary route R l ∈ {0, 1} N to determine whether a behavior token would be part of the behavior pathway or not. Each router takes the pre-order route R l−1 and user behavior tokens Z l−1 ∈ R N ×d of the block l − 1 as its inputs. All elements in the route are initialized by 1 and are updated progressively in training.
Foremost, to suppress the potential disturbance to the model caused by the local drifted interest (Figure 1), it is crucial to incorporate the global information in the route generation. We apply the average pooling to all the preserved behavior tokens routed by R l−1 , and produce the global sequential representation via a multilayer perceptrons (MLP) module. Then, we combine this global representation with the inputs and employ a residual connection to maintain the original input information. Finally, we feed them to another MLP layer to predict the probabilities of keeping or dropping the behavior tokens. The above procedure can be formulated as follows: where is the Hadamard product. For t ∈ {1, 2, · · · , N }, we let π t = [1 − α t , α t ] , where the logit α t denotes the probability that the t-th behavior token is kept for the behavior pathway.
Gumbel-Softmax sampling from π for router. Our goal is to generate the binary route from π. However, sampling from π directly is non-differentiable, and it will impede the gradient-based training. Thus, we apply the Gumbel-Softmax [15] technique to such sample. Gumbel-Softmax is an effective way to approximate the original non-differentiable sample from a discrete distribution with a differentiable sample from a Gumbel-Softmax distribution. Instead of directly sampling a keep-or-drop decision R l t for the t-th behavior token from the distribution π t , we generate it as: where G t = − log(− log U t ) is a standard Gumbel distribution, and U t is sampled i.i.d. from a uniform distribution Uniform(0, 1). To remove the non-differentiable argmax operation in Eq 3, the Gumbel-Softmax uses the reparameterization trick [15] as a differentiable approximation to relax one-hot where τ is the temperature parameter of the Softmax, which is commonly set to 1 in deep networks.
Hierarchical update strategy for router. The preliminary route R l , sampled from π, is not a final decision. In our design, once a token fails to be routed in a certain block, it would permanently lose the privilege to be part of the behavior pathway in the following feed-forward procedure, constituting a hierarchical pathway router strategy. Thus finally we formulate the route R l as the Hadamard product of R l and the pre-order route R l−1 in the block l − 1: Multi-head pathway attention. The standard self-attention mechanism retrieves sequential characteristics exploiting all behavior tokens, making the behavior pathway overwhelmed by the trivial behaviors. In the new pathway attention, the pathway router would be firstly applied to the input behavior tokens to route information. The pathway router would not pare down the number of tokens, but only the interactions between the off-pathway and on-pathway tokens, as these off-pathway tokens may also convey contextual information.
Specifically, for the query, key, and value in the pathway attention: the query is routed by the pathway router, to prevent the pathway from being overwhelmed and to force the pathway attention to attend to the behavior pathway; the key and value are the original input behavior tokens, to ensure that the contextual information from off-pathway behavior tokens can be captured as well: where m ∈ {1, 2, · · · , h} is the index of head in the multi-head self-attention; W l Qm , W l Km , W l Vm ∈ R d×d/h are transformation matrices learned from data. Finally, the outputs Z l m ∈ R N ×d/h 1≤m≤h of multiple heads are concatenated into Z l ∈ R N ×d . We use Z l , R l = Path-MSA (Z l−1 , R l−1 ) to summarize the above pathway attention. Its output is further transformed by Eq. (1) to form the final output of the l-th block Z l ∈ R N ×d .
Causality. In the prediction of the (t + 1)-th behavior, only the first t observable behaviors should be taken into account. To avoid a future information leak and ensure causality, a look-ahead mask is employed and all links between Q j and K i (j > i) are removed.

Prediction Layer and Training Objective
Prediction layer. In the final layer of our RETR, we calculate the user's preference score for the item k in the step (t + 1) in the context of user behavior history as p (i t+1 = k | i 1:t ) = e k · Z L t , where e k is the representation of item k from item embedding matrix E I , and Z L t is the output of the L-th RETR blocks at step t (L is the number of RETR blocks). Training objective. We adopt the pairwise ranking loss to optimize the model parameters as: where we pair each ground-truth item i t+1 with a randomly sampled negative item i − t+1 . In each epoch, we randomly generate one negative item for each time step in each sequence. This pairwise ranking loss is widely adopted in previous literature of sequential recommendation [16,41].

Experiments
We extensively evaluate the proposed Recommender Transformer on seven real-world benchmarks. Datasets.
Here are descriptions of the seven datasets: (1) Beauty, Sports, and Toys: these three datasets are three subcategories obtained from Amazon review [22] datasets. (2) Yelp [2]: Yelp is a dataset for business recommendation. We only use the transaction records after January 1st, 2019. (3) Tmall: Tmall contains users' shopping logs on Tmall online shopping platform, which is from the IJCAI-15 competition. (4) Steam [16]: Steam dataset is collected from a large online video game distribution platform. This dataset includes 2,567,538 users, 15,474 games and 7,793,069 English reviews from October 2010 to January 2018. (5) MovieLens: this is a widely used benchmark dataset for evaluating collaborative filtering algorithms. The version we use is MovieLens-1M, which includes 1 million user ratings.
We group the interaction records by users or sessions for all datasets and sort them by the timestamps in ascending order. We follow the operation in SASRec [16] and split the historical sequence for each user into three parts: (1) the most recent behavior for testing, (2) the second most recent behavior for validation, and (3) all remaining behaviors for training. During testing, the input sequences contain training behaviors and validation behaviors. We filter less popular items and inactive users with fewer than five interaction records. The statistics of the above seven datasets are summarized in Table 1. Beauty, Sports, Toys and Yelp have fewer average actions per user and item. MovieLens, Tmall and Steam have more actions per item.
Evaluation metrics. Following the previous literature [38,41,16], we apply top-k Hit Ratio (HR@k), top-k Normalized Discounted Cumulative Gain (NDCG@k) and Mean Reciprocal Rank (MRR) for evaluation. We report HR@10, NDCG@10 and MRR of the results. Besides, following the standard strategy in SASRec [16], we pair the ground-truth item with 100 randomly sampled negative items that the user has not interacted with. All metrics are calculated according to the ranking of the items and we report the average score.
Baseline methods. We compare our approach with seven baseline methods. The PopRec is a simple baseline that ranks items according to their popularity. Besides, we select four latest state of-the-art transformer-based models: SASRec [16], BertRec [28], SASRec+ [35] and SMRec [6]. We notice that SASRec [16] applies the transformer [4] into the sequential recommendation task; BertRec [28] introduces a Cloze objective loss for sequential recommendation; SASRec+ [35] improves SASRec with time embedding, which is inspired by the positional embedding in transformer; SMRec [6] adopts SASRec as the backbone, and it is associated with a novel self-modulating attention. These recent transformer-based methods have achieved state-of-the-art performance in many benchmarks. Besides transformer-based methods, we also adopt an RNN-based model GRURec [14] and a CNNbased model Caser [29] as baseline models. The GRURec [14] applies GRU to model item sequences and the Caser [29] applies convolution operations for sequential recommendation. All baseline methods are configured using default parameters of the original paper or optimal parameters which can produce the best results in a grid search.
Implementation details. Our model is supervised by the pairwise rank loss in Equ 7, using the ADAM [17] optimizer with an initial learning rate of 0.001. Batch size is set to 512. The maximum number of training epochs for all methods is set to 200. All hyperparameters are tuned on the validation set. The training process is early stopped within 10 epochs. Our RETR has L = 2 layers, and each layer has h = 4 heads and d is set to be 256. The maximum sequence length N is set to 200 for MovieLens-1m and 100 for the other six datasets. All experiments are repeated three times, implemented in PyTorch [23], and conducted on a single NVIDIA 3090 GPU.  [29], GRURec [14], BERTRec [28], SASRec [16], SASRec+ [35] and SMRec [6]) and our method on the Beauty, Sports, Toys, Yelp, MovieLens, Tmall and Steam datasets. We use HR@10, NDCG@10 and MRR as our metrics. For these three metrics, a higher value indicates a better performance.

Main Results
The results of different methods on seven datasets are shown in Table 2. First, the non-sequential recommendation method PopRec performs worse than sequential recommendation methods, indicating that capturing the sequential pattern is essential for sequential recommendation. Next, we can easily find that transformer-based models, SASRec [16], BertRec [28], SASRec+ [35], and SMRec [6], achieve better performance than RNN-based model GRURec [14] and CNN-based model Caser [29] on most datasets. It indicates that the transformer has a better capacity to capture sequential characteristics. The transformer-based models use the attention mechanism to capture the interaction information between all previous user behaviors, indicating that the attention mechanism is crucial for sequential recommendation.
Our RETR can achieve state-of-the-art performance by a large margin on most datasets compared with all baseline models. Specifically, our RETR achieves competitive performance on the Beauty, Sports, Toys, and Yelp. These datasets are sparse, containing less action information. Thus they have lots of noisy logged information. By effectively capturing the behavior pathway, our RETR is not affected by this trivial behavior information and captures the most informative behavior representation to achieve better performance. Besides, our RETR can also achieve better performance on the Tmall, Steam, and MovieLens, which contain a large number of actions. Note that under the Tmall benchmark, RETR gains 7% HR@10, 12% NDCG@10 and 14% MRR against the strongest baseline SMRec [6]. Under the Steam benchmark, our RETR also yields state-of-the-art performance, achieving 6% MRR gains compared with the SASRec. For the MoveLens benchmarks, our RETR also achieves the best performance among all competing baselines. These results strongly indicate that our RETR is consistently effective on both sparse and dense datasets.
Unlike these transformer-based baselines, our RETR adopts a pathway router to detect the most informative behavior tokens dynamically. These informative behavior tokens are denoted as the behavior pathway. Our RETR uses the behavior pathway and discards the less informative behavior tokens from the query. The substantial performance gains of our RETR indicate that focusing more on the behavior pathway enables RETR to capture sequential characteristics more efficiently and effectively than the vanilla self-attention mechanism, which considers all previous user behaviors.

Ablation Study
Effectiveness of each model component. In the left column of Table 3, we analyze the efficacy of each component in RETR on the Beauty dataset and have the following observations. First, we remove the pathway router module and randomly choose whether it can be maintained or dropped for each input behavior token. Removing the pathway router decreases the prediction performance a lot (MRR: 0.3067 → 0.2703), showing the necessity of learning behavior pathway effectively based on a data-dependent module. Second, discarding the hierarchical update strategy for the behavior pathway also decreases the prediction performance, suggesting that this strategy is crucial for RETR to get a more accurate behavior pathway. Notably, SASRec is the particular case of removing the pathway router, making routing decision 1 for all behaviors. Our RETR significantly outperforms the SASRec, indicating that capturing the behavior pathway effectively interacts with all previous behaviors.
Number of blocks. In the right column of Table 3, we adjust the number of blocks for RETR on Beauty. We find that the performance first increases rapidly with the growth of the block number and achieves the best performance at L = 2. We perform a similar grid search on other datasets.  Effectiveness of temperature. In the left column of Table 4, we analyze the efficacy of different temperatures for Gumbel-Softmax sampling in RETR on the Yelp dataset. We observe that the perfor-mance first increases rapidly with the growth of the temperature and achieves the best performance when τ = 0.8, while the performance degenerates a lot when τ > 1. The temperature τ softens the softmax with τ > 1. However, when τ → ∞, the Gumbel-Softmax distribution p τ (y t ) → 0.5 becomes more smooth, leading to the maximum uncertainty. To make the sampling results more convincing, we apply the temperature calibration τ < 1 during training to avoid overconfident predictions. These results show that Gumble-Softmax sampling with lower temperature (τ < 1) avoids overconfident predictions, leading to better performance.
Evaluation on efficiency. The efficiency is compared between SASRec, SASRec+ and SMRec on the Yelp dataset. The computation cost is measured with gigabit floating-point operations (GFLOPs) on the self-attention module with position encoding. Meanwhile, the model scale measured with parameters is also presented. As shown in Table 4, our RETR has almost the same number of parameters or GFLOPs, compared with SASRec, indicating that our pathway router is a light-weight module. Our pathway attention does not bring more costs. It's worth noticing that the parameter scales and GLOPs of other competing transformers (apart from SASRec) are larger than RETR, but our RETR achieves higher performance. This result shows that our RETR is more efficient and effective than other competing transformer-based models.

Visualization
Setups. We also provide qualitative visualizations for our RETR, SASRec [16] and SMRec [6]. Technically, we use the GradCAM [26] to generate behavior heat maps of the output of the last layer in each model. A random user's historical behaviors in the Steam dataset are given in sequential order (row-wise). The last item is an RPG game The Dwarves in the test set. We provide attention heatmaps on the last ten positions at the last ten time steps in Figure 3. As shown in Figure 3, two main behavior pathway characteristics exist in this sequence: (1) Casual Behavior Pathway: RPG games are randomly clicked by the user, while the user has a continuing interest in RPGs.
(2) Drifted Behavior Pathway: The user has recently been interested in adventure games but chooses the RPG at last. We hope that a sequential recommendation model can figure out the exact behavior pathway.
Visualzation results. As shown in Figure 3, we can find that the baseline method can not mitigate the drifted behavior pathway and capture the casual pathway adaptively. It seems that these models can be misled by the recent Correlated Behavior Pathway. In the third row of Figure 3, the SASRec focuses more on the recent adventure games and overlooks the user's casual interest in RPGs, which indicates the SASRec is easy to be overwhelmed by the drifted behavior pathway. The SASRec cannot effectively mitigate the drifted behavior pathway and capture the casual behavior pathway. For SMRec, even though it adopts a time weight mechanism. However, only using the time weight mechanism can not help it capture the practical behavior pathway. On the contrary, as shown in the second row of Figure 3, our RETR can capture the casual behavior pathway precisely with higher scores, and avoid being misled by the drifted behavior pathway. This visualization results strongly show that our RETR can capture the behavior pathway dynamically for each user.

Conclusion
A sequential recommender is designed to make accurate recommendations based on users' historical behaviors. The sequential recommendation system has benefited many practical applications such as online advertising and social media. However, the users' behaviors are dynamic and come in a continually evolving manner due to various reasons. We conclude these sequential characteristics as the behavior pathway. Previous models cannot capture the behavior pathway dynamically. We propose the Recommender Transformer (RETR) with a novel pathway attention mechanism to tackle these challenges. The pathway attention develops a pathway router to dynamically get the behavior pathway for each user and capture the evolving patterns. Our RETR achieves state-of-the-art performance on seven real-world datasets for sequential recommendation.