{'fraction_non_alphanumeric': 0.037823716492503406, 'fraction_numerical': 0.012693094048159928, 'mean_word_length': 4.85, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Sequential Embedding Approach for Item Recommendation with Heterogeneous Attributes

Attributes, such as metadata and profile, carry useful information which in principle can help improve accuracy in recommender systems. However, existing approaches have difficulty in fully leveraging attribute information due to practical challenges such as heterogeneity and sparseness. These approaches also fail to combine recurrent neural networks which have recently shown effectiveness in item recommendations in applications such as video and music browsing. To overcome the challenges and to harvest the advantages of sequence models, we present a novel approach, Heterogeneous Attribute Recurrent Neural Networks (HA-RNN), which incorporates heterogeneous attributes and captures sequential dependencies in \textit{both} items and attributes. HA-RNN extends recurrent neural networks with 1) a hierarchical attribute combination input layer and 2) an output attribute embedding layer. We conduct extensive experiments on two large-scale datasets. The new approach show significant improvements over the state-of-the-art models. Our ablation experiments demonstrate the effectiveness of the two components to address heterogeneous attribute challenges including variable lengths and attribute sparseness. We further investigate why sequence modeling works well by conducting exploratory studies and show sequence models are more effective when data scale increases.


Introduction
In recent years recommendation with implicit feedback (also known as item recommendation) has been increasingly active in research Hu et al. (2008); Rendle et al. (2009); Johnson (2014); He et al. (2016), and is applied in various applications like e-commerce Linden et al. (2003), social networks Chen et al. (2009), location Liu et al. (2014), etc. After observing user activities such as clicks, page views, and purchases-often called implicit feedback-the goal is to recommend to each user a ranked list of items that he might prefer. Item recommendation remains a very challenging task. Most users have very limited interactions with extremely small portions of items and often no negative feedback Hu et al. (2008). User-item interactions are implicit and hard to interpret Hu et al. (2008); Liu et al. (2015). Different collaborative filtering methods are heavily exploited to alleviate these challenges Hu et al. (2008); Rendle et al. (2009); Rendle and Freudenthaler (2014); Usunier et al. (2009); Weston et al. (2010); Shi et al. (2012); Li et al. (2016).
Content-based approaches are important alternative methods to collaborative filtering. Increasingly prevalent attributes (such as metadata or profile) of users and items, e.g., LinkedIn user profiles and Yelp business metadata, offer new opportunities and trigger active research. To make use of the attributes, low-rank factor models such as matrix co-factorization Fang and Si (2011); Saveski and Mantrach (2014), tensor factorization Karatzoglou et al. (2010); Bhargava et al. (2015), and hybrid matrix factorization Shmueli et al. (2012); Kula (2015) are developed to jointly learn latent factors of attributes as well as users and items. While these matrix factorization extensions provide a simple mechanism to incorporate attributes, in practice they are not adequate to fully leverage attribute information due to challenges such as heterogeneity and sparseness. Moreover, these approaches are based on bi-linear models and thus are too restrictive in their model flexibility to capture complex dependencies between different model components. Figure 1: An illustrating example where a job seeker interacts with a sequence of job posts. Rich heterogeneous attributes are associated with both the job seeker and job posts. The system is asked to recommend new posts to this user. Sequential dependency is a prominent example of such dependencies. Item sequential dependencies have proved useful in recommender systems Rendle et al. (2010); Cheng et al. (2013); Kapoor et al. (2015); Du et al. (2015). Recently, flexible sequence models based on RNNs and Word2Vec are used to model item transitions explicitly and show better prediction performances than factor models in E-commerce Grbovic et al. (2015), music Vasile et al. (2016), and videos Hidasi et al. (2015). However, these sequence models either work with no attributes or one or two domain-specific features. Dependencies between attributes and items and within attributes are largely ignored. This also prevents better understanding of user behaviors and further improvement to system performance.

Internet and IT
Attributes are often heterogeneous and come in different domains and data types. For example (as in Fig.1), attributes may include a user's age, location, education, and a job's industry, title description, employment, online tags; their data types include real numbers, categorical tokens, text tokens, etc. To develop efficient approaches to incorporating attributes, we investigate three major challenges in real scenarios-variable lengths, sparseness, and sequential dependency. Variable lengths: Due to the natures of different attributes, users or items may have different lengths of attributes. A user may have academic degrees in more than one discipline; a job post may vary in the number of tags; missing values are very common. It is inefficient to simply combine the observed attributes. Sparseness: The possible attributes of items include tags from the Internet, text tokens, demographic information, etc. The entire attribute vocabulary can be very large, but each attribute appears only a limited number of times. To add difficulty, a large part of the attributes may be irrelevant to our task of interest. These factors pose further challenges to model regularization. Sequential dependency: In cases where a user interacts with a sequence of items (or services), attributes may help encode user sequential behaviors. As evidence, attributes of nearby items in a sequence may overlap or share common characteristics. For example as in Fig.1, job post attributes collectively suggest the user's current interest in a junior-level position in an IT-related area. It is desirable to make use of such attribute dependencies.
In this work we propose a novel model, Heterogeneous-Attribute Recurrent Neural Networks (HA-RNN), to address the above challenges. HA-RNN combines sequence modeling and attribute embedding in item recommendation. Different from conventional RNNs, HA-RNN develops novel embedding techniques and integrates them into sequence modeling. In particular, • It develops a hierarchical attribute combination mechanism to deal with variable lengths of attributes.
• To battle against attribute sparseness, the model uses attributes in the output layer and shares the parameters with the input layer to offer additional model regularization.
• HA-RNN takes the union of identity and attributes as a sequence element and is able to capture the global sequential dependencies between items as well as between attributes.
The model is trained end-to-end and learns task-driven attribute representations Experiments are conducted on two large-scale datasets for implicit recommendation. Our methods give significant improvement compared to state-of-the-art baseline models. Results show that attribute embedding and sequence modeling are both essential to improving performance. Particularly, the novel output attribute layer design helps boost recommendation accuracy. We qualitatively demonstrate our model's ability to discover attribute semantic structures. Finally, further analysis is conducted to understand how item sequences influence recommendation.
The contributions of this work are summarized as follows: • We study the problem of item recommendation with heterogeneous attributes. Challenges to incorporate these attributes include variable lengths, sparseness, and sequential dependencies.
• We propose an approach that combines attribute embedding and recurrent neural networks to incorporate attribute in item recommendation. We develop novel techniques to address attribute heterogeneity challenges.
• We perform empirical studies on two real-world datasets and show the effectiveness of our approach. Our model significantly outperforms the state-of-the-art models. Detailed analysis indicates the critical roles of our different model components.

Approach
In this section we introduce our proposed model HA-RNN. After defining the problem formulation and notation, we first present an existing approach based on RNNs in item recommendation. Then we describe our proposed model that combines sequence modeling and attribute embedding and deals with heterogeneity challenges. Figure 2: The overview of the proposed HA-RNN model. The base recommendation model is recurrent neural networks where item sequence (with the user) is fed as input and the model is trained to predict the next item. The representations for the input (Q U and Q I ) and for computing item scores in the output layer (Q I ) are combination of identity and attributes. To compute Q I , embedding of the multi-hot attributes (e.g., φ M 1 ) are first averaged before combined with categorical ones (e.g., φ C 1 , φ C 2 ; identity is regarded as categorical attribute in the figure) and numerical ones (e.g., φ N 1 ). The same item representation is shared in both input and output layers for enhanced signals and model regularization. Computation of Q U is omitted in the figure.

Problem Formulation and Notation
We are given a user set U , an item set I, and their interactions S = {(u, i, t)|u ∈ U, i ∈ I} where t records the time that interaction (u, i) takes place. Let A U , A I denote the attribute set associated with users and items. Parameters associated with users, items, user attributes, and item attributes are denoted by e U ∈ R |U |×d , e I ∈ R |I|×d , φ U ∈ R |A U |×d , and φ I ∈ R |A I |×d , respectively-given the model dimensionality d. Subscripting means taking one corresponding slice of the matrix. For example, e I i ∈ R d denotes a vector representation of item i; φ U a ∈ R d (a ∈ A U ) denotes embedding of one user attribute a. Superscript U or I is omitted when there is no ambiguity. In the context of a sequence, integer subscripts denote sequence positions.
A recommender system needs to return a scoring function g such that s(i) = g(i|u, U, I, S, A U , A I ) captures user-item preferences where an item preferred by a user to another receives a higher score. The recommendation then takes the highest scored item (as shown in Eq. 1).
In model training the scores are compared to the ground truth observations. Different loss functions are developed Hu et al. (2008); Rendle et al. (2009);Weston et al. (2010). In this work we use cross-entropy classification loss as in Grbovic et al. (2015); Hidasi et al. (2015).

Identity Embedding via Sequential Recommendation
We begin with an existing sequence recommendation approach Hidasi et al. (2015). First, items seen by a user u are sorted chronologically as a sequence (u : i 1 , i 2 , ..). Then, as in a sequence generation problem, an RNNs-based model learns identity embedding of users and items by fitting the model to predict the next appearing item given all the observed ones. The model formulation can be written as the following, where f is an RNN-cell (e.g., LSTM, GRU); s(i n ) is computed from the inner product between latent state h n and item matrix w ∈ R d×|I| . 1 In this work we focus on q and w. As the input layer and the output layer of networks, they connect sequence models with data observations. In Hidasi et al. (2015), the input vector q simply takes item embedding, and w is a separate set of item parameters from e I . While the model is able to capture sequential dependencies between items, it does not use attributes at all as neither q nor w involve attributes.

Joint Attribute Embedding
We first extend the model to incorporate attributes in the input layer-both item identity and attributes contribute to the input representation. Particularly, where attr(i) returns the set of attributes of item i. When user attributes are also available, we similarly define q u and thus have q n = q u + q in . This model, however, is inefficient in practice. It suffers from the mentioned challenge-variable attribute lengths. For example, when an item has more attributes than others, its second summand in Eq. 5 tends to have larger magnitude, and the estimation becomes harder. As another example, if an item has more attributes for one type but less for another, the mismatch is harmful when the model tries to compare representations of two items.

A Hierarchical Attribute Combination
To deal with the challenge, we propose a division of heterogeneous attributes into three distinct kinds: categorical (C), multi-hot (M), and numerical (N ). In particular, • Categorical attributes are those with exclusive labels, which means an object can only have one value for that type.
• Multi-hot attributes are often descriptive, and an object is accompanied by one or more attribute values of a type.
• Numerical attributes can have continuous values in real numbers with their specific domain interpretations.
In the example in Figure 1, categorical attributes include the job post identity, industry, employment requirement, and location. Title text tokens, tags here or in other examples like movies tags are multi-hot attributes. User "age" here belongs to the numerical group.
We point out that multi-hot attributes-which can have more than one value for each attributeand missing values are the causes of variable lengths of attributes. 2 With the division and a bit of abuse of notation C, M, N , we design a hierarchical way to combine the attributes and have the representation as follows: where C j (i) (or M j (i), N j (i)) returns item i's jth categorical (or multi-hot, numerical) attribute(s). 3 n C (or n M , n N ) denotes the total number of attribute types belonging to C (or M, N ). q u is computed similarly. Compared to (5), multi-hot attribute embedding is no longer summed together. Rather, the mean of embedding within each type of multi-hot attribute is computed before embeddings across types are combined. In this way, values of one multi-hot attribute are regarded as a "single attribute", and we have a better control of the scales of input vector q.

Shared Attribute Embedding in Output Layer
With (2)(3)(6), RNNs embed attributes as part of model input. However, as most conventional RNNs, attributes are not involved in the output prediction stage. To address the attribute sparseness challenge, we first explore incorporating attributes in the output layer in addition to the input layer. Intuitively, use of attributes in these two network components enhances attribute signals in model training.
We extend the output layer to involve attribute embedding in computing item scores. Particularly, we discard w and compute the score for item i by where ⊕ ∈ {mean, max}. First, attribute preference scores are computed by dot product between model latent vector h and attribute vectors. Then the attribute scores within each multi-hot type take mean or max. Intuitively, mean-pooling suggests that each attribute value contributes equally, while max-pooling suggests that one particularly favorable attribute value dominates. Finally, scores across different attribute types are summed (or averaged) to produce item scores. Note that item identity and attributes now appear both in input and output layers of the model. A natural question is whether or not we should use separate embedding parameters for the two components. Different from common practice such as in Word2Vec, we decide the input and output layers share the same embedding parameters, i.e., This reduces the number of total parameters and, more importantly, adds additional constraints to embedding parameters. We expect this model regularization to help the model achieve better generalization.
HA-RNN. With (2)(3)(6)(7)(8), we have our complete model HA-RNN. As in Figure 2, attributes and identities are coupled in the sequence model through both input and output layers. Given an item sequence, the model first looks up and combines the attribute embeddings and then feeds the representation into the networks. Attribute parameters are updated together with network parameters via back-propagation. Compared to the sequential recommendation model in Sec. 2.2, HA-RNN treats the union of identity and attributes as a sequence element and tries to capture sequential dependencies in both identities and attributes. Compared to simple attribute incorporation in Sec. 2.3, HA-RNN carefully designs input and output layers to deal with heterogeneity challenges.

Related Work
Attribute incorporation. Low rank factorization models are extended to incorporate attributes in recommendation. Matrix co-factorization (MCF) Fang and Si (2011) Recently, more flexible models (e.g., topic models and pre-trained neural networks) have been used to model attributes of text Bansal et al. (2016); Kim et al. (2016), vision He and McAuley (2015), and music Van den Oord et al. (2013). The learned embedding from these flexible models are then fixed and used in a downstream matrix factorization recommender system. As a comparison, our approach targets heterogeneous types of attributes; meanwhile, it learns attribute embedding in a task-driven fashion to avoid domain adaption issues. Graph-based algorithms like graph random walk are used to learn attributes Christoffel et al. (2015); Levin et al. (2016). They construct graph nodes as identities and attributes, and graph edges as events and attribute relations. Attributes and identities are then embedded in the same space.
Recently, Word2Vec training techniques are tailored to recommendation models. It shows scalability and efficiency in E-commerce product recommendation Grbovic et al. (2015). Vasile et al. (2016) extends Grbovic et al. (2015) to incorporate metadata in a multitask learning fashion and shows improved performance on a music dataset. Comparisons In designing our approach to incorporate attributes, we have the following considerations: 1) task-driven: attribute embedding is learned end-to-end for recommendation tasks; 2) heterogeneous: the model can handle more than a few attribute types; 3) flexible in model capability: we want the model to go beyond matrix factorization; 4) sequential: it has the ability to model sequential dependencies. We compare our approach to other related works in Table 2 and highlight our approach combining all these properties. While we do not argue these properties are in general the most desirable, we find in our empirical study they are helpful to guide to achieve good performance.

Experiments
This section discusses how we conduct extensive experiments to validate the effectiveness of our approach. First, we introduce the experimental setup. Then we present the recommendation accuracy results comparing different variants of HA-RNN and compare these with other state-of-the-art models. This is followed by a qualitative analysis of learned attribute embedding. Finally, we review our studies on how the sequence component helps recommendation. The results demonstrate how the embedding techniques and sequence modeling help achieve recommendation accuracy improvement. We validate our approach on two large-scale datasets: 1) job recommendation from XING 4 ; 2) business reviews at Yelp. Both datasets contain rich attributes associated with users and items, in addition to user-item interactions. We describe the datasets in detail below.
XING is a dataset adapted from the RecSys Challenge 2016 Abel et al. (2016). It contains about 14 weeks of interaction data between 1,500,000 users and 327,002 job posts on XING. The interactions are user online activities including click, bookmark, and reply. Train/test splitting follows the official challenge setup where the interactions in the first 12 weeks are used as training data, and the interactions for a set of 150,000 target users in weeks 13 and 14 are used as test data. Our task is to recommend a list of job posts to the target users; these are the posts that users are likely to interact with (either click, bookmark, or reply) in the last two weeks.
Rich attributes are associated with users and job posts. Attributes are anonymized tokens in different fields such as career levels, education disciplines, industries, locations, work experiences, job roles, job titles, etc. For example, an attribute token may represent a HR discipline, or consulting; it may also represent a word in a job title. The detailed attribute information is listed in Table 3. We remove attributes that appear less than two times. The size of the remaining attribute vocabulary is 57,813.
Yelp is a dataset that comes from the Yelp Challenge. 5 There are 1,029,433 users and 144,073 businesses. Interactions here are defined as reviews. We work on recommendation related to which business a user might want to review. Following the online protocol in He et al. (2016), we sort all interactions in chronological order, taking the last 10% for testing and the rest for training. There are 1,917,218 reviews in the training split and 213,460 in the testing split.
Business items have attributes including city, state, categories (e.g., "meditation centers," "thai foods"), hours, and features (e.g., "valet parking," "good for kids"). Similarly, we remove rare attributes with less than two appearances. The attribute vocabulary size is 37,573. Dataset statistics are listed in Table 4.

Evaluation
We assess the quality of recommendation results by comparing the models' recommendations to ground truth interactions, and report precision, recall, MAP, and normalized discounted cumulative gain (NDCG). For precision and recall metrics, we report results at positions 2, 10, and 30. We report MAP and NDCG at location 30. Note that we report scores after removing historical items from each user's recommendation list on the Yelp dataset because in these scenarios (Yelp reviews), users seldom re-interact with items. This improves performance for all models, but does not change relative comparison results.

Models
We compare our models with models that show variant abilities in incorporating attributes and utilizing sequence in item recommendation. Baseline non-sequence models POP and WARP Weston et al. (2010) and sequence models CBOW Grbovic et al. (2015) and RNN Hidasi et al. (2015) do not incorporate attributes. Non-sequence model HMF Kula (2015) and sequence approach A-RNN incorporate attributes. We also experimented with different variants of our proposed HA-RNN to investigate the roles of different model components. We detail the models and hyper-parameter tuning below. Hyper-parameters tuning and early stopping are done on a development dataset split from training for all models. Selected hyper-parameters do not land on tuning boundaries.
• POP. A naive baseline model that recommends items in terms of their popularity.
• CBOW. A sequence recommendation approach Grbovic et al. (2015) based on Word2Vec techniques. We used our own implementation. We use window size 5, dropout rate 0.5. m in CBOW is tuned in {1,2,3,4}. We also experimented with its alternative approach based on Skip-gram. CBOW works better. We omit our results for brevity.

Effectiveness of multi-hot treatment
To begin, we investigate the effectiveness of multi-hot treatment in our attribute embedding. We call our combination (6) discussed in Section 2.4 HET. We compare this to (5) (we call MIX) which simply takes the average embedding of all attribute values. Recommendation accuracies and training time are reported in Table 5. In terms of recommendation accuracy, our multi-hot treatment HET brings significant improvement to HA-RNN. In both datasets, HET significantly outperforms MIX. Particularly in Yelp, the improvement is dramatic (41% relative gains on MAP and 34% on NDCG). This may be due to the fact that variable lengths of attributes are really harmful to flexible models such as RNNs. Our HET deals with this heterogeneity and provides within-category normalization. This turns out to be very critical in model regularization.
In terms of training time, we see that it takes much longer to train HA-RNN with MIX than HET. This is expected as the MIX model has to additionally adjust the embedding scales while HET does not. We stopped the training process of HA-RNN-MIX after 6 days. To evaluate the effectiveness of the output attribute embedding layer, we explore four variants of HA-RNN: 1) 'no attributes': No attributes, but identity is used-this degenerates to a recommendation model based on standard RNNs as in Hidasi et al. (2015); 2) 'only input': attributes used at input layer; 3) 'only output': attributes used at output layer; 4) 'input+output': attributes used at both input and output layers. Experiment results are reported in Table 6, from which we make two observations. First, HA-RNN performs poorly on both datasets when no attributes are used ('no attributes'). On XING, the result is even worse than that of non-sequence model NHMF (see NHMF HET variant in Table 5). On the contrary, with attribute embedding ('input+output'), the HA-RNN scores are significantly boosted. These results suggest that sequence modeling alone is not good enough, and attributes do help to improve accuracy.
Second, the output layer design for attribute embedding turns out to be very important. On XING, only incorporating attributes at the output layer ('only ouput') gives better results than only at the input layer ('only input'). Moreover, it works best to learn attributes at both layers ('input+output'). On Yelp, embedding attributes either at input layer or output layer alone seems hardly helpful; however, by embedding attributes at both layers we manage to significantly improve the score. This verifies our assumption that the output layer embedding brings additional regularization.
To validate the effectiveness of sharing attribute parameters in both input and output embedding layers, we run HA-RNN with separate sets of parameters φ, e and φ , e . We report validation perplexities in Table 7. 'only output' and 'in+out (shared)' are models we have discussed in Table 6. 'in+out (sep)' denotes the new model configuration.
In Table 7, 'in+out (sep)' clearly underperforms 'in+out (shared)' in both datasets; on XING dataset, it is even worse than 'only output' model. Additional attribute parameters in the output layer lead to worse generalization. This suggests the necessity of sharing parameters across input and output layers.

Performance comparisons
We compare HA-RNN with other state-of-the-art models and report the results in Table 8. We interpret the table as follows. First, attribute embedding is vital in XING-the scores of HMF and HA-RNN are significantly better than those of WARP, CBOW, and RNN. Recommendation does benefit from sequence approaches as RNN performs better than WARP; HA-RNN does better than HMF. Second, sequence approach is critical in Yelp as we see CBOW, RNN, and HA-RNN clearly beat WARP and HMF. Finally, the best accuracy is achieved when HA-RNN combines both attribute embedding and sequence modeling. The improvements are significant. On XING dataset, CA-RNN single model relatively improves Precision@2, Recall@30, MAP, and NDCG by 29%, 25%, 36%, and 30%; on Yelp dataset, the improvements are 12%, 16%, 16%, and 14%.

Other Sequential Recommendation Models
We want to empirically study the effect of sequence modeling on recommendation performance. In order to do that, the same embedding techniques in HA-RNN are applied to skip-gram and CBOW models (we call the new models HA-SG and HA-CBOW). In this way, we assume the new models have similar capabilities in incorporating attributes but differ in their abilities in capturing sequential dependencies. The results are shown in Table 9. We make three observations below. First, non-sequence models HMF and NHMF have similar performances. Second, HA-CBOW clearly beats HMF and NHMF on both datasets although it has a similar bi-linear model formulation as in HMF and NHMF. It shows HA-CBOW does benefit from its sequence-based training. Finally, HA-RNN outperforms HA-CBOW significantly. We attribute it to its stronger ability in sequence modeling and its flexibility.

Learned Attribute Embedding
We give our qualitative analysis on the embedding learned by HA-RNN. We take the learned attribute embedding and plot the 2-D visualization (Fig. 3). On XING, we observed an interesting hierarchical clustering effect of categorical attribute embedding. First, the same type of attribute embedding tends to cluster. For example, user career levels such as student/intern, manager, executive, etc., form a cluster; executives, and senior executive understandably stand a bit farther from the rest. Second, embedding clusters across types tends to stay close when they have close semantic meaning. For example, (anonymous) attributes of industry and discipline types are closely related and thus have close distances. This is encouraging since we started from heterogeneous   attributes and managed to infer their semantic structures. On Yelp, we take a look at the learned embedding of multi-hot attribute "categories." 6 We see that embedding reflects the correlation of business types in the reviews. For example, we locate the top nearest neighbors of juice bars & smoothies (distances are computed in the embedding space): macarons, cardio classes, meditation centers, cafes, wine tasting room .... This suggests that people who review a business of type juice bars & smoothies should likely review businesses of type cardio classes or wine tasting room. The information might be used in cases where a new business is recommended to a Yelp customer. Some other examples (including those from XING) are listed in Table 10.

Sequence Modeling Studies
The observation in Table 8 that HA-RNN outperforms HMF and RNN suggests the benefits in the explicit modeling of sequential properties of both item and attributes. We are further interested in the following two questions.
Should we sample items from item sequences or not during model training? When applying sequence modeling to the recommendation problem, we implicitly assume that sequence or order provides additional information beyond that provided by item frequency alone. Is this a valid assumption and how strong is the sequential dependency? In the modeling perspective, HA-RNN has two major differences compared to HMF: explicit sequence modeling and flexible non-linear function mapping. It is possible that the advantages of HA-RNN mainly come from the latter. Meanwhile, data augmentation by sampling is widely used as a regularization. It may help improve the performance by sampling items.
What if dataset size changes? The current experiments are performed on two fixed-size datasets. We are wondering if the better performance of sequence models is an artifact of small dataset sizes? In other words, does the non-sequence model catch up with sequence models when more observations are given?
To investigate these two questions, and in general how item sequences influence the performance of HA-RNN, we conducted additional experiments on XING and report two of our findings below. 7 Sequence vs. frequency In the new experiments with HA-RNN, we generated new training data through sampling subsequences in which items in a user's item sequence were dropped out with certain probability. The sampling procedure is repeated to augment item sequence. For example, an item sequence {i 1 , i 2 , i 3 , i 4 , i 5 } might now become three subsampled sequences: {i 1 , i 3 , i 4 }, {i 2 , i 3 , i 5 }, and {i 1 , i 2 , i 4 , i 5 }. New sequences were used in the training, but on average item appearance frequency tended to be unchanged in data with more and more augmented subsequences due to unbiased sampling. Experimental results with the new generated training data are shown in Table 11. "Original" denotes dataset based on the complete sequence (i.e., no sampling augmentation). x N denotes the manipulated data set obtained by sampling and augmenting subsequence proportional to N times. First, increasing subsequence sampling leads to decreasing scores (from x 1 to x 8 ); second, the original data set (full sequences) gives the best score. These results suggest that it is harmful to break item sequence even though the item frequency is maintained the same. This verifies our assumption that item sequences indeed provide additional information than frequency alone.
Performance vs. sequence number Another way to test our assumption is to increase (or decrease) the number of observed sequences and to compare performance between non-sequence and sequence models. While both non-sequence and sequence models observe the same total number of user-item pairs, sequence models might have a chance to extract more useful information from the data if our Table 11: HA-RNN relative scores on original and "manipulated" XING datasets. "Original" denotes the complete sequence. x N denotes the manipulated data set obtained by randomly sampling and augmenting subsequence proportional to N times.

Sampling Original
x 1 x 2 x 4 x 8 Score 1.0 0.97 0.94 0.86 0.84 assumption is true. We train models NHMF and HA-RNN on XING and still evaluate on target users. We gradually increase the training observations from those of target users to those of a super-set of users until all users are included. The percentages of total interaction used for four experiments are 20%, 44%, 76%, and 100%, respectively.
Results are reported in Figure 4. We see that across scores returned by different metrics, 8 HA-RNN improves steadily when the data scale increases (more sequences observed). On the contrary, NHMF in general does not improve. This can be explained by our assumption that useful information is encoded in the sequence: recommendation accuracy improves when more sequences are observed. This is not the case when more independent user-item pairs are observed. Our proposed HA-RNN approach successfully utilizes this helpful information and benefits from that. From another perspective, it suggests that sequence models may even be better suited when we have larger-scale recommendation from implicit feedback. Based on this, we would advocate the use of sequence modeling in large scale recommendation setting.

Conclusion
In this paper we explore the effectiveness of combining heterogeneous attribute embedding and sequence modeling in recommendation with implicit feedback. To this end, we build a neural network framework to incorporate attributes and apply sequence models to embed attributes and to perform recommendation. Through empirical studies on four large-scale datasets, we find that rich discriminative information is encoded in heterogeneous attributes and item sequences. By combining attribute embedding and flexible sequence models, we are able to capture information and improve recommendation performance.