{'fraction_non_alphanumeric': 0.03347280334728033, 'fraction_numerical': 0.006859181013786954, 'mean_word_length': 4.812039067171616, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
KQGC: Knowledge Graph Embedding with Smoothing Effects of Graph Convolutions for Recommendation

Leveraging graphs on recommender systems has gained popularity with the development of graph representation learning (GRL). In particular, knowledge graph embedding (KGE) and graph neural networks (GNNs) are representative GRL approaches, which have achieved the state-of-the-art performance on several recommendation tasks. Furthermore, combination of KGE and GNNs (KG-GNNs) has been explored and found effective in many academic literatures. One of the main characteristics of GNNs is their ability to retain structural properties among neighbors in the resulting dense representation, which is usually coined as smoothing. The smoothing is specially desired in the presence of homophilic graphs, such as the ones we find on recommender systems. In this paper, we propose a new model for recommender systems named Knowledge Query-based Graph Convolution (KQGC). In contrast to exisiting KG-GNNs, KQGC focuses on the smoothing, and leverages a simple linear graph convolution for smoothing KGE. A pre-trained KGE is fed into KQGC, and it is smoothed by aggregating neighbor knowledge queries, which allow entity-embeddings to be aligned on appropriate vector points for smoothing KGE effectively. We apply the proposed KQGC to a recommendation task that aims prospective users for specific products. Extensive experiments on a real E-commerce dataset demonstrate the effectiveness of KQGC.


INTRODUCTION
Internet services, including e-commerce and streaming, have been growing up rapidly in recent years. Recommender Systems (Rec-Sys) are indispensable technologies for enhancing user experience of these services. Therefore, how to fully leverage rich sources of data and improve RecSys have been one of the most critical challenges for companies in the field. RecSys have been widely studied in both research and industry, and various approaches have been proposed over the years. Most of them have relied on collaborative filtering (CF), which infers user's latent preferences based on the cooccurrence in historical interactions. Traditional CF-based methods, such as user-based model [3,16], item-based model [1,13,14] and matrix factorization [10], generate user's and item's features from an interaction matrix. However, they cannot incorporate side information, resulting in poor performance in the cold start problem [20,22].
Graph representation learning-based approaches have gained popularity in recent years due to the ability to handle with both high-order relations in historical interactions and side information. In particular, Graph Neural Networks (GNNs) have shown prominent performance in RecSys based on their ability of modelling complex settings and scalability. User-item bipartite graphs have been employed for GNNs traditionally [18,23,25,29,30], while utilization of knowledge graphs (KGs) have been also considered lately [20,22,24]. Leveraging KGs has two main advantages. Firstly, it can handle attributes relationships (e.g., hierarchical dependency of product genres in e-commerce). Secondly, extracting attributes as entities (nodes) in a KG allows two nodes having the same attributes to be connected through the attribute entities. As a result, GNNs can capture high-order relations by sharing attributes more efficiently [22]. The representative models combining KG and GNNs (KG-GNNs) are KGCN [20] and KGAT [22]. KGCN is a meta-path based GNN, which generates embeddings of entities and relations in the KG with knowledge-aware attention-based GNN layers. KGAT consists of two layers: Knowledge Graph Embedding (KGE) and GNN layers, and it generates node embeddings by applying GNN layers to the KGE obtained from TransR [11]. Here, the two layers are trained simultaneously.
We argue that homophily holds in graphs that emerge in recommendation tasks, and aggregation (smoothing) is the most effective function of GNNs there as Simple Graph Convolution [26] has shown. In this paper, we propose Knowledge Query-based Graph Convolution (KQGC), which enhances KGE with smoothing. KQGC relies on the simple principle of smoothing, and it is a KG-GNN model that has different perspective to KGCN and KGAT. Therefore, there are three distinctions between existing KG-GNNs and KQGC: 1) KQGC consists of both KGE model and graph convolution model. KGE is pre-trained, and subsequently the KGE is fed into graph convolution model. i.e., the two models are trained separately in contrast to KGAT. 2) KGCN and KGAT aggregates only neighbor entity embeddings, while KQGC aggregates knowledge queries, which are sums of entity and relation embeddings. That can alleviate irrelevant smoothing. 3) KGCN and KGAT apply nonlinear function when updating embeddings, while KGCN does not. As a results, property of the pre-trained KGE is preserved.
Rakuten group has one of the largest E-commerce platform in the world. In particular, AIris Target Prospecting (AIris TP), which is a framework that finds prospective users for Rakuten business clients, has been playing an important role in this E-commerce platform. Companies like Facebook runs one of the world's most powerful advertising platforms, which is called "Facbook Lookalike Audience". Rakuten AIris TP serves the same purpose to its clients. Therefore, we aim to improve the performance of AIris TP in this paper. We apply KQGC to the Airis TP, and evaluate the effectiveness.
The main contributions of our work are organized as follows: • We propose a new model combining KGE and graph convolution called KQGC, which enhances KGE with smoothing effects of graph convolution. It provides a new perspective of combining KGE and GNNs. • Experimentally, we show the effectiveness of KQGC and its smoothing effects on real E-commerce dataset.

RELATED WORK
Graph representation learning (GRL) for RecSys is roughly categorized into three approaches: knowledge graph embedding (KGE), graph neural networks (GNNs) and combination of the two. In this section, we describe each of them and clarify relation between them and the proposed model.

Knowledge Graph Embedding
Knowledge graph (KG) is a directed heterogeneous graph that represents a fact, where nodes and edges represent entities and relations between two entities, respectively. It is formulated as a set of triples {(ℎ, , )|ℎ, ∈ E, ∈ R}, where ℎ, , are heads, tails and relations. For instance, the fact that shoes A is sold by Shop B is represented as a triple ( ℎ , , ℎ ). The main goal of KGE is to map enitities and relations in KGs to embeddings (vectors).
There are two representative categories of KGE models: bilinear (semantic matching) model and translation-based model [5]. Bilinear model is a tensor factorization model, which generates embeddings of entities and relation matrices that can reconstruct relational adjacency matrix of the KG. RESCAL [12], DistMult [28] and ComplEX [17] are representative bilinear models. In translationbased model, embeddings of entities and relations are initialized with random values, then they are updated so that distance between embeddings of positive triples (e.g., || ℎ + − || 1,2 in TransE) is minimized. TransE [2] is the first translation-based model, and there are several variants of it such as TransR [11] and TransD [8].
In the context of RecSys, recommended items or users are determined by evaluating similarity score between user-and itemembeddings obtained by KGE models. More specially, in the case that we recommend items to a user, top-items that have high score (e.g., inner product between the user and items with purchase relation ( + ℎ ) ) are selected as the recommended items. Our model employs translation-based models in the pre-training phase, and subsequently smooths the pre-trained KGE with graph convolution. Consequently, our model strengthens the homophily of KGE.

Graph Neural Networks
Graph Neural Networks (GNNs) are neural network models for graphs, which encode node/edge embeddings by leveraging both node/edge features and graph structure. There are three types of GNNs: recurrent GNNs, spectral-based convolutional GNNs and spatial-based convolutional GNNs (spatial-based ConvGNNs) [27]. In particular, many variants of spatial-based ConvGNNs have been proposed actively in recent years [6,7,9,15,19] due to the flexibility of modeling and scalability. Generally, GNNs are formulated with the message passing (MP) framework [4], which consists of two steps: aggregation and update steps. Firstly, the aggregation function aggregates neighbor node features of destination nodes, where we term updated nodes destination nodes. The update function then updates the destination node features by combining the aggregated neighbor node features with themselves. Here, neural networks are employed in both the aggregation and update functions usually.
GNNs have shown their remarkable capabilities for RCs as well as classification tasks over the years. User-item bipartite graphs, which are constructed with user-item interactions, are utilized as the input graphs in the literature. Furthermore, user demography features and item attributes are utilized as node features in contrast to KGE which treats the features as entities (nodes). Aggregating neighbor node features is regarded as collecting collaborative signals. Hence GNN-based approaches for RecSys are the CF-based models with side information, that possess various expansions for complicated modeling and large scale applications.
On the other hand, a paper has argued that nonlinearity in GNNs is not necessary needed for several tasks. The authors have proposed Simple Graph Convolution (SGC) [26], which is a graph convolution without activation function, and shown its competitive performance compared to the state of the art GNNs. The simplification improves interpretability of the model and reduce computational complexity. We believe that the fact holds in tasks related to homophily and recommendation tasks are involved in there. In this paper, we employ a new graph convolution without activation function, which allows the property of the traslation-based models to be preserved.

Combination of KGE and GNNs
Two different GRL approaches are described in the above, while combination of the two has been also studied in recent years. In [22], the authors introduce collaborative knowledge graph (CKG), which merges a user-item bipartite graph and a KG of items, and apply KGAT to it. KGAT is a model that combines KGE and GNNs. Firstly, it generates KGE with TransR [11], and subsequently the KGE is fed into knowledge-aware attention-based GNN layers. TransR and GNN layers are trained simultaneously with the loss function of TransR and a pair-wise loss function. KGCN [20] is a mata-path based GNN model, which consists of only GNN layers designed for KGs. In KGCN, embeddings of entities and relations are initialized with random values as with normal KGE models, then the embeddings are updated with knowledge-aware attention-based GNN layers. Nodes that have the same attributes are connected each other in KGs. That allows GNNs to capture high-order relations explicitly, resulting in enhancing effectiveness of aggregation of GNNs. In fact, they reaches the state of the art performance in several recommendation tasks.
However, the two models aggregates only entity embeddings in GNN layers, and that possibly incurs irrelevant smoothing (described in Section 3.2). To alleviate this issue, the proposed model aggregates knowledge queries, which consists of both entity and relation embeddings. The proposed framework has two components: KGE and a linear graph convolution, and they are trained separately in contrast to KGAT.

METHODOLOGY
We now introduce the proposed Knowledge Query-based Graph Convolution (KQGC), which improves pre-trained KGE with smoothing effect of graph convolution. Figure 1 shows the overall framework, which consists of two models: translation-based KGE model and KQGC. The whole procedure for applying KQGC to downstream tasks is as follows: 1) a knowledge graph (KG) is fed to the knowledge graph embedding (KGE) model, then KGE for all the entities and relations are trained in unsupervised learning; 2) the pre-trained KGE is input to KQGC as initial node/edge features, and the KQGC is trained with downstream tasks. Note that the KGE is fixed when KQGC is trained, i.e., the two models are trained separately. We hereafter describe the details of the two models (Section 3.1 and 3.2), and then introduce a use case of KQGC in Rakuten group (Section 3.3).

Translation-based Knowledge Graph
Embedding model In this work, we employ TransE as the translation-based KGE model. In TransE, the embeddings of all the entities and relations are initialized with random values, then they are updated by the gradient descent method based on the max-margin loss function L KGE .
where (ℎ, , ) is positive triples, which actually exist in the KG, (ℎ ′ , , ′ ) is broken triples, is the margin, (ℎ, , ) is the score function, and [·] + = (0, ·). The broken triples are generated by replacing either head-or tail-entity of the positive triples with an entity randomly chosen. The score function of TransE is as bellow.
where ℎ , and , are the embeddings of head entities, relations and tail entities, respectively. In Eq. (2), either L1 or L2 norm is usually used, we employ L1 norm. Minimizing Eq. (1) with this score function (Eq. (2)) leads a relation between embeddings of positive triples ℎ + ≈ in the same vector space ( Figure 2). From the other perspective, sum of a head and relation embeddings (the red arrow in Figure 2) reaches a near point to the related tail embeddings, therefore it can be regarded as a query to visit tail entities related to the head entity with the relation. In this paper, we term it knowledge query. KQGC, which is described in the next section, aggregates the knowledge queries obtained by TransE for aligning neighbor node embeddings in the vector space.

Knowledge Query-based Graph Convolution
After training of KGE is completed, the trained KGE is fed into KQGC and smoothed by it. KQGC follows the MP framework and consists of two steps: aggregation and update.
In terms of aggregation, KQGC aggregates knowledge queries instead of neighbor node features. Aggregation of knowledge queries is depicted in Figure 3. The knowledge query from a source node to the destination node with a relation, which is aggregated when updating the destination node feature, is defined as where and are embeddings of the source node and the relation, and the triple ( , , ) exists in the KG. In the case that the KG is a directed graph and we have only embedding of one direction relation, we obtain that of the reversed relation by multiplying the embedding by -1. One of the main efficacy of aggregating knowledge queries is alignment of node embeddings in the vector space, which is shown in the right-hand side of Figure 3. We think of the case that updates embeddings of user A and item A as an example. For simplicity, let the aggregation be simple mean operation here. Assuming that each of the entity and relation embeddings is mapped on the unit circle as shown in Figure 3, the aggregated (averaged) neighbor node features for both user A and item A end up being similar (Figure 3 (a)), even though they do not have any nodes shared in common. On the other hand, the aggregated knowledge queries are aligned to near points to the embeddings of the destination nodes, resulting in preservation of similarity obtained from KGE model (Figure 3 (b)).
In the update phase, KQGC combines the aggregated knowledge queries with the target node embedding using linear transformation. We do not leverage non-linear function in this phase in order to preserve the property of translation-based model when multiple layers of KQGC are stacked. SGC has shown adequate capability of graph convolution without nonlinear function for several tasks related to homophilicity. Intuitively recommendation task is included to the tasks, and we believe that absence of nonlinear function is not critical. We provide formulations of aggregation and update in below.

Aggregator.
Here we propose mean-and attention aggregators. We redefine notation of knowledge queries for multiple KQGC layers, as follows.
where indicates -th layer, Q is the -th query from source to the destination with relation , and are the -th embeddings of the source and relation, respectively. KGE obtained from TransE is utilized as initial embeddings of all the nodes and relations, i.e., 0 = , 0 = . For the second and subsequent ones, node and relation embeddings outputted from the previous layer are input to the next layer. The two types of aggregators are formulated as follows.
Mean aggregator. This aggregator simply takes the average of neighbor knowledge queries, which is formulated as, where Q is the -th message of knowledge queries, N ( ) is the set of neighbor nodes of node , R ( , ) is the set of relations between and .
Attention aggregator. Rather than aggregating neighbor knowledge queries equally, attention aggregator considers the importance of each knowledge queries and aggregates them with different weights.
In this paper, we provide two types of attention aggregators, which are formulated as, ( where ( , , ) is normalized attention coefficient, ∈ R 2 is the trainable parameter, is dimension of the embeddings, and ∥ denotes concatenation. The first one (Eq. (8)) employs inner product between the knowledge query and the destination node to calculate attention coefficients, which is the same as the attention mechanism of KGCN. Knowledge queries that are closer to the destination node in the vector space are aggregated with higher weights in this attention mechanism. The second one (Eq.(9)) has trainable parameters and they allow us to automatically design how to aggregate knowledge queries with appropriate attention coefficients, based on the loss function of the downstream tasks. The formulation is inspired by the self-masked attention mechanism of GAT.

Update.
After aggregating knowledge queries, new embeddings of all the nodes and relations are obtained by combining the destination node embedding with the aggregated knowledge queries. We employ the following update rule.

Figure 3: Aggregation of knowledge queries in KQGC
where and are the updated embeddings of the target nodes and all the relations, which are inputs of the next layer, and are the -th trainable weight parameters. This update rule simply sums up the destination node embedding and the aggregated knowledge queries, and then map them to new vector space with linear transformation. The transformation is also applied to the relation embeddings so that translation between entites and relations can be constructed in the new vector space as well. Therefore we do not employ nonlinear function in this formulation.

Applications of KQGC in Recommender Systems of Rakuten Group
Rakuten group has various services and each service employs different model, therefore it is impossible to replace all the models with end-to-end fashion of KQGC. Instead, we attempt to improve the services by generating general user embeddings that can be used across various Rakuten services. A framework for applying KQGC to various Rakuten services is depicted in Figure 4. In this framework, KQGC and models in different services are completely separated for the portability. User embeddings obtained from KQGC are utilized in common as input features of each model with the baseline features. Types of the baseline features depend on types of services, which include user demographic features or user's purchase history.
In this paper, we focus on a Rakuten service called Target Prospecting (TP), and evaluate KQGC on the task of TP. The task of TP is to find prospective customers for clients (e.g., shops or brands) and prepare a list of top k active Rakuten users. The definition of active users is: the users who buy from Rakuten Ichiba regularly within average buying period. Currently, we have more than 40 millions active users. This paper employs XGBoost, which is a binary classifiers, as the service-side model. i.e., user embeddings obtained from KQGC and the AIris baseline features (refer to Section 4.2) are input to the binary classifier, and it classifies prospective customers with true/false. We hereafter describe how to construct a knowledge graph for KQGC and how to train KQGC for our AIris TP task. 3.3.1 How to construct knowledge graphs. Rakuten group has the largest E-commerce platform called Rakuten Ichiba, which stores rich data of user behavior and item attributes in there. Purchase behavior of users is intuitively informative for not only predicting perspective users for clients but also capturing user preferences, which can be generally used for other services. Hence we leverage the data on Rakuten Ichiba 1 for constructing KGs, and obtain the general user embeddings based on the KGs. The procedure of constructing KGs is as follows: 1) collect seed users, who are picked up based on purchase history from Rakuten Ichiba users. 2) collect all the items purchased by them. 3) extract demography features of the seed users and attributes of the items. 4) construct a KG: connect users and items with purchase history; attributes are extracted as entities and they are connected to users and items. After creating the KG, we apply the framework of KQGC to it, then we finally obtain the general user embedding for the seed users.

Training.
As our goal here is to obtain user embeddings from the KQGC, there is no decoder in the downstream task. Therefore, we employ the following unsupervised loss function for training KQGC.
where (U, I) ∈ P is the positive pairs between users and items with purchase relation, (U, I ′ ) ∈ P −1 is the negative one, ( U , I ) is the score function especially for user-item interaction, U and I are user and item embeddings, purchase is purchase-relation embeddings. I ′ denotes negative items, which are generated by replacing the positive item with an item randomly chosen in the positive pairs. Here we employ L1 norm for Eq. (13).

EXPERIMENTS
In this section, we conduct experiments to evaluate the performance of our model using the same metric (PR-AUC) which is being used in AIris TP for evaluation. The KQGC model has been implemented to the Rakuten AIris TP to provide effective target prospecting. We first explain the AIris TP system overview and experimental settings. Then, we evaluate the effectiveness of our model with five brand datasets (the name of the brands have been kept anonymous ) by comparing it with the baseline model. Finally, we briefly discuss the effectiveness of the model in the real target prospecting scenario.

Overview of the Rakuten AIris TP
In this section, we briefly describe the Rakuten Airis TP framework. Figure 5 shows the dashboard of AIris TP for clients. A client/marketer can upload a file of users (to find out similar users) or she can use "Seed ID Manager " to select appropriate seed users for the task or retrieve the previous purchasers of a product/brand using "Purchaser ID Finder". There are two more options for travel (Travelers ID Finder) and books (Book Readers ID Finder) related targeting. After uploading the seed users using one of the options mentioned above, clients can start a job for finding lookalike customers for their business purpose.

Experimental set-up
Datasets. In AIris TP, for a specific product/brand we pick up the users who already bought that product/brand in previous months and then AIris TP extracts potential customers for a product/brand who have high probability to buy the product/brand in next month. We adopt five cosmetic brands as the AIris clients in this experiment and evaluate models based on whether AIris can extract accurate potential customers for each of the client from given users, with the features obtained from the models.
To treat this problem as binary classification, we arrange both positive users (label 1) and negative users (label 0) for each client, in the datasets of TP. Positive users are those who purchased at least one item of the client in a certain time-window, i.e., positive users correspond to the accurate potential customers in this evaluation. On the other hand, negative users are those who purchased no item in the same time-window as positive users. Ratio between positive and negative users is set to around 1 : 3 for each client. In terms of data split, we collect train/validation and test users in different time-windows to remove possible data leakage. 20K users are collected in two months from Jul. 1st, 2021 to Oct. 31st, 2021 as training and validation data sets of each client, and they are randomly divided into 6 (training) : 4 (validation). 5K test users for each client are collected in Sep. 1st, 2021 -Sep. 31st, 2021 (one month). If the number of positive users for a client in the timewindow is less than 20K or 5K, the number is used instead of 20K or 5K. Purchase behavior for constructing KG is collected in Feb. 1st, 2021 -Jul. 31st, 2021, following the way described in 3.3.1.Statics of the datasets of TP and the KG used for training KQGC are shown in table 1 and table 2, respectively.
Baselines. To demonstrate the effectiveness, we compare our proposed model with the current model and on production and other KGE models.
• Baseline (AIris TP): This is the current model on production (see Fig. 5). It is based on XGBoost model. This model

Results
Our goal is to improve PR-AUC for various clients. Hence we hereafter emphasize the average of the PR-AUCs for all the clients. Note that user embeddings and baseline features are shared among all the clients, on the other hand, AIris is optimized for each client independently. The best PR-AUCs over 10000 epochs are shown in table 3.  In table 3, the values in parentheses are improvement rate (%) compared to the baseline and the other values are PR-AUCs. The highest PR-AUC among different input features is marked in bold for each column. Table 3 shows that KQGC outperforms both the baseline and the original TransE except for Brand D, and KQGC(mean) reaches the best PR-AUC for the average of all the clients. Fig. 6 shows evolution of PR-AUC of the averaged improvement rate with epochs. From Fig. 6, we can observe that KQGC(mean and attn1) achieves more stable performance than others. That mean aggregator's performance is the best and attn2 aggregator's one is unstable possibly because the embedding module and the prediction one are separated. The attention mechanism of KQGC are automatically designed by the loss function of pre-training (Eq.13), therefore it is difficult to construct an attention mechanism that is suitable for TP. Figure 6: Evolution of the average of PR-AUC improvement rate with epochs

Model Deployment
The AIris TP is a well-known Rakuten target prospecting framework. Rakuten group has started this service from the year of 2018.
Periodically we evaluate the deployed model's performance and improve the model. The proposed KQGC is in the testing phase for deployment on our AIris TP framework. The major advantage of AIris TP is: it allows the marketers/clients to perform the target prospecting without training the entire model from scratch when we have a new task (job), e.g., find the potential users with similar behavior patterns to new brand's existing users. The tentative customer conversion rate (CVR) is expected to go much higher in comparison with the previous standard solution.

CONCLUSION
In this paper, we have introduced a novel model for recommender systems that leverages both KGE and graph convolution. KQGC smooths knowledge graph embedding (KGE) obtained from translationbased models with the aggregation of neighbor knowledge queries, and helps adjacent entities have similar embeddings. Empirical result on Rakuten group's real data shows that KQGC outperforms current AIris TP baseline models including TransE, which is the original translation-based model, and we also observed smoothing KGE is effective for improving performance on recommendation tasks.
In future work, we will explore the following three things. (1) Deeper analysis of our model; In this work, our model employs one convolutional layer, and it is evaluated on only our private dataset. To confirm the effectiveness of our model more strongly, we will evaluate our model with more than two layers, and compare it with several SoTA models on various public datasets. (2) Incorporation of side information; continuous attributes such as image-or textembeddings are difficult to be extracted as entities on a KG. Hence we need to investigate how we can effectively combine KGE with those continuous attributes as inputs of KQGC. (3) Generality of our model; Our ultimate goal is to apply user embeddings obtained from KQGC to various services. As this paper focuses on prospect recommendation task, we will evaluate the user embeddings on different services and investigate the generality as our next research direction.