{'fraction_non_alphanumeric': 0.031607567690041916, 'fraction_numerical': 0.00788489860654809, 'mean_word_length': 4.728971962616822, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Cross-Market Product Recommendation

We study the problem of recommending relevant products to users in relatively resource-scarce markets by leveraging data from similar, richer in resource auxiliary markets. We hypothesize that data from one market can be used to improve performance in another. Only a few studies have been conducted in this area, partly due to the lack of publicly available experimental data. To this end, we collect and release XMarket, a large dataset covering 18 local markets on 16 different product categories, featuring 52.5 million user-item interactions. We introduce and formalize the problem of cross-market product recommendation, i.e., market adaptation. We explore different market-adaptation techniques inspired by state-of-the-art domain-adaptation and meta-learning approaches and propose a novel neural approach for market adaptation, named FOREC. Our model follows a three-step procedure -- pre-training, forking, and fine-tuning -- in order to fully utilize the data from an auxiliary market as well as the target market. We conduct extensive experiments studying the impact of market adaptation on different pairs of markets. Our proposed approach demonstrates robust effectiveness, consistently improving the performance on target markets compared to competitive baselines selected for our analysis. In particular, FOREC improves on average 24% and up to 50% in terms of nDCG@10, compared to the NMF baseline. Our analysis and experiments suggest specific future directions in this research area. We release our data and code for academic purposes.


INTRODUCTION
Nowadays online shopping in many countries is a part of people's daily lives. While online shopping brings several benefits and comfort to both users and vendors [20], it comes at the risk of overwhelming users with virtually unlimited options to choose from. Recommender systems are key in dealing with information overload, helping not only users finding interesting items, but also vendors finding the right customer for their products. E-commerce companies often operate across markets; for instance Amazon 2 has expanded their operations and sales to 18 markets around the globe. 3 This brings both opportunities and challenges.
While it is typical that several local e-commerce companies operate in every country, the presence of an international e-commerce company, like Amazon, eBay, and Etsy can benefit users even more if these companies can utilize the experience and data gathered across several markets. Using cross-market data however comes at a risk of assuming one-solution-fits-all and applying the same algorithms that are developed for and trained on large and data-rich markets, such as the U.S. [19], to small and data-scarce markets.
The key challenge is that data, such as user interaction data with products (clicks, purchases, reviews), convey certain biases of the individual markets [5]. Algorithms that are optimized for a certain market learn various biases and distributions of the data [44]. Therefore, the algorithm trained on a source market, are not necessarily effective in a different target market [11], since utilizing the vast amount of data from a large market to improve the performance on low-resource markets comes at the risk of importing the wrong data distributions. For example, assume iPhone is the most popular smartphone in the U.S. (source market), while Samsung is the most popular smartphone brand in Germany (target market). Importing user preference from the U.S. market would yield to recommending iPhone in Germany more often than Samsung, which clearly is a wrong choice. Hence, even though there is a myriad of information to learn from a source market, careful adaptation of data is required.
The significance of Cross-Market Recommendation (CMR) has been pointed out in the literature [43]. However, small progress has been made in this area, mainly due to a lack of experimental data. To this end, we construct a large-scale real-life product recommendation dataset, referred to as XMarket, drawn from 18 markets in 11 languages. To develop this dataset we crawled Amazon marketplaces around the globe, locating and including in the dataset the same products within different markets. Moreover, we analyze certain statistical properties and trends amongst multiple markets and product categories where we highlight the existence of crucial differences across different markets.
In this paper, we focus on the user-item interaction data through ratings and study the problem of recommending relevant products to users in relatively resource-scarce markets by leveraging data from similar, richer in resource auxiliary markets. Our hypothesis is that data from one market can be used to improve performance in another. For this purpose, we first introduce and formalize the problem of cross-market product recommendation, i.e., market adaptation. In order to solve the problem of CMR, we explore market-adaptation baselines inspired by domain-adaptation and meta-learning approaches. Then, we propose a novel neural approach, named FOREC, consisted of a three-step procedure -pretraining, forking, and fine-tuning -in order to fully utilize the data from an auxiliary market to boost the product recommendation performance in the target market. More specifically, FOREC learns a general recommendation system based on two markets (i.e., source and target) and employs a forking procedure by adding a market specific sub-network to the head of the model and freezing the bottom part in order to adapt the general internal representations to the target market. We conduct extensive experiments studying the impact of market adaptation on different pairs of markets. FOREC demonstrates robust effectiveness, consistently improving the performance compared to competitive baselines on 7 target markets we selected for our study. In particular, FOREC improves on average 24% and up to 50% in terms of nDCG@10, compared to the NMF baseline. Our analysis and experiments provide many insights on the CMR and suggest specific future directions in this research area. In summary, the main contributions of this paper constitute: • Collecting a real-world pragmatic large-scale cross-market and cross-lingual product review dataset. • Performing analysis of cross-market behavioral biases pivoting on the fact that our dataset includes the same items shared across different markets. In particular, we study how differently users in different markets interact with the same products. • Proposing a novel neural architecture for market-adaptation and demonstrating the effectiveness of the model through extensive experiments. We adapt various Cross-Domain Recommendation (CDR) and meta-learning approaches for Cross-Market Recommendation (CMR). • Analyzing the performance of the model, considering various setups and conditions to provide further insights.

RELATED WORK
This study is related to CMR and CDR, as well as meta-learning approaches. In this section, we briefly review the research done in these domains.
Cross-domain and cross-market recommendation. Research on CMR and CDR aims at improving the system's effectiveness based on the external data that is available from other markets or item categories. While having the same goal, the two tasks differ in various aspects, each bearing their own challenges. Particularly, in CDR the general assumption is that the model learns from interactions of overlapping users 4 in different domains (e.g. categories in product search) with the aim of improving the recommendation on the target domain items, using help from the source domain items. For CMR the situation is reversed: interactions of different set of users in the source market are leveraged to boost the recommendation for users in the target market. Here, we assume that the items are shared among different markets. Im and Hars [19] conduct an experiment in two product domains, aiming to answer the question "does a one-size recommendation fit all?" where they observed that the performance of Collaborative Filtering (CF) is highly affected by the information-seeking mode of the users. Depending on the product domain, users adopt different strategies and therefore the system would not fit to all domains. Lu et al. [34] later argue that transferring all the knowledge from source domain into the target domain may harm the recommender due to some inconsistencies and propose a criterion for selecting the consistent part of knowledge to be transferred to the target domain. Elkahky et al. [10] apply domain adaptation using user behaviorbased features for learning latent space. Rafailidis and Crestani [40] propose a collaborative ranking model with a weighting strategy that controls the influence of user preferences from auxiliary domains. Zhao et al. [52] use reviews to transfer user preference at aspect-level as a cross-domain recommendation framework. Krishnan et al. [28] propose using contextual invariances across domains to leverage data from a dense domain to improve learned representations in other sparse domains. Different from these works, in this paper we investigate the existence of different behavioral biases across different markets where unlike CDR, the items are the same across different markets but the users are different. Further research has aimed at mitigating these biases and transferring knowledge from one domain to another [18,30]. Unsupervised domain adaptation [14] has also inspired various cross-domain recommender systems in recent years [23,31,47,50]. CDR has been used specifically for mitigating the cold-start problem in a number of studies [13,21,22,24,37,49,52].
While there exists much work on domain adaptation, marketadaptation is relatively unstudied. CMR has attracted attention in music recommendation [11,43] where Ferwerda et al. [11] analyze and study music diversity across countries and propose to use country-based diversity measurements for system evaluation. Roitero et al. [43] studied user behavior in 21 different markets on Spotify and highlight the need for market-specific algorithms, as opposed to a global algorithm. We take one step further in this direction by expanding our study to various item categories in e-commerce where users purchase items (rather than having a monthly subscription) and express their opinion and experience with item in the form of ratings and reviews.
Meta-learning. The goal of meta-learning is to train a model on multiple tasks, such that it can rapidly adapt to a new task after seeing a small number of new training samples [46]. In the context of recommendation systems, meta-learning has been used for several problems, including but not limitted to recommender algorithm selection [8,9,35], cold-start problem [29,45], and retraining the model [51]. For meta-learning of deep neural networks, a general and powerful technique, Model-Agnostic Meta-Learning (MAML), has been proposed that can be directly applied to any learning problem and model [12]. MAML framework is widely used in recommendation literature. For example, Lu et al. [33] use it on heterogeneous information networks to address the cold-start problem. Others used MAML to train a recommender which performs reasonably good enough both for cold-and warm-start users [4,29].

DATA COLLECTION & ANALYSIS
In this section, we first describe in detail the data collection process and provide statistics for the collected data. Then, we analyze the data and highlight important characteristics and similarities across different markets.

Data Collection
We describe our new dataset, called XMarket, and provide details on how we generated it. We constructed the XMarket item and review collection on top of a large-scale publicly-available Amazon dataset [16,36] 5 . The Amazon Product data [36] includes millions of item reviews collected from the Amazon U.S. marketplace in various categories. The dataset was collected in 2014 and later updated in 2018 [38]. We used this dataset as a seed to initiate our crawl. We located the same items that appear on the U.S. market in other markets, by matching the items' unique identifiers (aka. ASIN's) on all available Amazon markets. In particular we have crawled data from the following markets: Saudi Arabia (sa), Singapore (sg), Australia (au), United Arab Emirates (ae), Turkey (tr), Japan (jp), India (in), Spain (es), U.S. (us), China (cn), Germany (de), Netherlands (nl), France (fr), Brazil (br), Canada (ca), Mexico (mx), Italy (it), United Kingdom (uk). Our main criterion for including an item in the collection process was its popularity on the us market. Our decision was motivated by the idea of having a high-resource market (i.e., us) that would provide a wealth of data to other markets. Therefore, we discarded all items with less than 20 reviews in the past two years, as we did not consider them rich enough to be useful in other markets. In our preliminary analysis, we noticed that in most cases, if an item exists in another market its ASIN is the same. Therefore, we fed our crawler with the ASIN's that we collected from the U.S. market. In doing so, we collected cross-market metadata information for over one million items and collected about 52 million multilingual reviews. Among the existing multi-lingual review datasets, we find the Multilingual Amazon Reviews Corpus (MARC) [26] the most similar 5 https://jmcauley.ucsd.edu/data/amazon/ to XMarket. MARC consists of multi-lingual reviews extracted from different Amazon marketplaces, however, the scale of the dataset is much smaller. In particular, they do not cover all Amazon marketplaces and categories, whereas XMarket covers a wide range of categories in all 18 Amazon marketplaces. Moreover, XMarket includes rich item and review metadata (e.g., reviewer ID, item description, and related items) that can be utilized to pursue various research directions. We also found another similar dataset named as Amazon Customer Reviews Dataset 6 providing a collection of reviews from five marketplaces dated from 1995 to 2015. We notice that a vast majority of the provided data is only from United States and the provided meta-data is limited to only product title and reviews whereas our data covers more number of marketplaces with a full meta-data information (including product text, images, also bought, similar items). In addition, our product reviews are more recent. To the best of our knowledge, no other cross-market multi-lingual recommendation dataset with such a wide coverage of markets and categories exists in the community. Table 1 summarizes some of the main characteristics of XMarket. It provides a cross-lingual e-commerce dataset of 16 shopping categories in 11 languages. We crawled data for ∼300K unique items across all markets, which resulted in ∼4M cross-market items. Also, Fig. 1a shows the distribution of items in each market. We see that the Canadian (ca) and Mexican (mx) markets have the most items in common with the U.S. market (us), which is expected due to the long-lasting presence of Amazon in these countries and their vicinity to us. Due to space considerations and similarity in results, in the remainder of the paper we analyze and discuss a subset of markets and categories. Our experiments and model evaluations are based on the Electronics category with statistics presented in Table 2. We see in the table the main characteristics of the studied marketplaces in terms of recommendation data, such as number of items, users, and ratings. We observe a relatively high number of users as well as items in the U.S. market, making it the most sparse market in our dataset.

Cross-Market Analysis
Our goal in this section is to analyze and demonstrate the similarities and differences across markets. Having in mind that the markets share the same set of products, we analyze how people in different regions interact with these items to uncover behavioral characteristics and biases.
Distribution of ratings. Several reasons may influence users to purchase a product such as users' financial status, culture, and companies' marketing strategies. Therefore, we study the difference in product rating (as a signal of product purchase) among different markets. We plot the distribution of product "purchase" in Fig. 1b. The dominance of us is obvious in this figure, with the highest median in all categories. We see that different markets exhibit different distributions across categories. In general we see that Electronics is most popular category among different markets. We also observe that the Home & Kitchen category shows a different trend compared to other categories, perhaps because such items are more regionally dependent.
Distribution of rating starts. We are interested in finding out if the same items are rated differently in each market. Also, if the differences happen across categories. We find significantly different behavior in giving rating stars among markets and categories, as shown in Figure 1c. We see a greater tendency of giving higher rating to items in mx market, whereas for in market, we see an opposite behavior. Interestingly, we observe a relatively similar trend in all categories where for example the median rating in de market is always higher than that of us and fr market, but slightly lower than the uk market. This clearly shows a general bias in user rating behavior, which should be taken into account when developing algorithms such as rating prediction.
Market similarity. Based on the observations that we had from Figure 1, we estimate the users' purchase similarity between markets. For a pair of markets in the same category, we build item purchase count vectors and compute cosine similarity between the two vectors. In Figure 2 we plot the similarities of all market pairs for two categories. Interestingly, we observe the highest rate of similarity between de, fr, and uk, highlighting the similarities in European markets. On the other hand, we see that the American countries do not share much in common. As we see us, ca, mx exhibit low similarity, which is surprising. Perhaps this is due to strong existence of local vendors in this category. In particular, we see the lowest similarity between de and mx.
Remarks. Overall, from our analyses it is evident that users in each market exhibit different behavior. These differences could be due to various reasons such as cultural biases or different marketing strategies adopted by companies. Another reason could simply be the popularity of Amazon as an e-commerce marketplace in different countries and how long it has been doing business in each country. For instance, we saw a very high similarity between us, ca, and mx in terms of common items that exist in the three markets, however, when it comes to product purchase data, we see very little similarity. This indicates that even though Amazon has a big influence on the e-commerce market in these countries, users act differently. With the existence of obvious differences and similarities, but at the same time having a mix of data-rich and scarce markets, learning from auxiliary markets is not trivial and requires careful development of market adaptation techniques.

PROBLEM STATEMENT
Assume we are given a set of parallel markets as = {M 0 , · · · , M }. Let M 0 be the base market with the set of items I 0 = { 1 , · · · , }. For the base market, one could assume the market with long-lasting existence offering the super-set of items and rich user-item interaction data. For example, with our XMarket settings, the us market can be thought of the base market and others such as de or in are considered the parallel target markets. We assume that I ⊂ I 0 for 1 ≤ ≤ . Depending on the provided parallel markets, , the base market could change or there might be no base market. With any of these settings, a union set of items in all the parallel markets could be defined as I 0 , satisfying our assumption.
For a given target market, M , let the set of market's users as U = { 1 , · · · , }. Generally, a user can interact with different markets, but for simplicity, we assume that the set of users in each market are mutually disjoint with any other parallel market. The problem of market adaption is to use any of the parallel markets provided, M ∈ − {M } as an auxiliary market to improve the quality of items recommended to users of the target market, i.e. U . It is straightforward to use more than single auxiliary market. However, we focus on single auxiliary market and leave the other variations as the future work. For our experiments, we either augment with only us market or any of the parallel markets and report the results. Automatically selecting the most suitable parallel auxiliary market is another interesting problem that is out of the scope of this study Item Emb  Figure 3: The general schema of our FOREC recommendation system. For a pair of markets, the middle part shows the market-agnostic model that we pre-train, and then fork and fine-tune for each market shown in the left and right. and we plan to explore on that direction as our future work.

FOREC: A CMR SYSTEM
Here, we explain our proposed CMR system, named FOREC. A general schema of our model is presented in Fig. 3. We show an example model architecture for a pair of markets, de and us. The training phase for the FOREC system includes three ordered steps (i.e., pretraining, forking, and fine-tuning) that we explain in the following. Note that FOREC is capable of working with any desired number of target markets. However, for simplicity, we only experiment with pairs of markets for our experimental evaluation.

Pre-Training: Market-Agnostic NMF
In this step, we aim to train a recommendation model that is marketagnostic, in the sense that all the model parameters are shared across markets and easily adaptable to every target market. This provides a generalized recommendation performance and a set of internal latent representations that are suitable for each individual market. Having such internal representations maximize the reusability of parameters translating into minimal effort on target market adaptation. To this end, we exploit the Model-Agnostic Meta-Learning (MAML) framework [12] from the few-shot learning literature.
The general neural architecture we use for our pre-training step is presented in middle part of Fig. 3. This architecture is first introduced by He et al. [17] and widely used in the literature. Here, we summarize the Neural Matrix Factorization (NMF) deep network before explaining our learning paradigm across markets. NMF model fuses two sub-networks namely Generalized Matrix Factorization (GMF) and Multi-Layer Perceptron (MLP). Both GMF and MLP subnetworks are trained with the data individually and then fused using the NMF model architecture. For each user and item, a onehot vector is constructed and fed to the user and item embedding layers of GMF and MLP networks, respectively. Note that GMF and MLP sub-networks keep their own embeddings space (i.e., no embedding sharing). Let the user and item latent vectors be p and q , respectively. The GMF network simply perform an element-wise product between the vectors and uses a linear single-layer feedforward network to calculate the prediction, i.e. h (p ⊙ q ). Here, h denotes the weights of the output layer. For individual training of GMF, the prediction^G MF is calculated using an activation function over the output layer's output. This implements a generalized matrix factorization network.
For the MLP sub-network, the user and item vectors are concatenated and fed to a deep feed-forward network for learning the interactions between user and items, i.e. 1 = p ⊕ q . Every layer of the deep network, takes the output of the previous layer z and calculates the output, i.e. W z −1 + b . W and b denote the weight matrix and bias vector of each layer. Here, we use ReLU as activation function and calculate^M LP for individual training. The NMF model simply initialize the network with individual pretrained parameters and for the output layer it concatenates the output layer of GMF and MLP with a hyper-parameter determining the trade-off. For the parameter initialization of our pre-training for market-agnostic NMF model, i.e. initialization, we simply concatenate the data from all target markets (a pair of markets in our study) and train the model. For the loss function, L, we use the binary cross entropy between the target and the model output.
Algorithm 1 lines 2-10 present our market-agnostic NMF pretraining. The general framework for meta-learning considers a probability distribution over tasks. Given the highly imbalanced training data that each market offers and our final goal to learn a set of generalized network parameters that works for each individual market, we consider equal task sampling across markets. To this end, we simply iterate over each market instead of sampling randomly (line 3). During the meta-training with -shot setting, for each market the model is trained such that it can adapt itself with only samples from the target market. This pushes the model parameters such that they broadly become applicable to each individual market. For this purpose, we sample two sized batches of the user-item interaction and perform adaptation and evaluation (lines 4 and 7).
Considering the NMF model with parameters , with the adaptation step on market M the model parameters become ′ . With a single gradient update, ′ = − ∇ L M NMF( ) in which is the NMF model's learning rate (line 5-6). The meta-learning optimizes the NMF parameters across markets with the following meta-objective.
min ∑︁ The meta-optimization across markets are calculated using the evaluation user-item interactions on each market with a metalearning step size, . This updates the original model parameters such that few gradient steps can tune the parameters to a specific target market (line 9).

Forking: Market-Specific Model Construction
After obtaining the general internal representations using our market-agnostic pre-training step, we conduct a series of operations for preparing a model that is market-specific for the desired target market. We call this step "forking" mainly due to the sharing of bottom parts and initialization of the middle parts of the marketspecific model with the pre-trained model. The general schema of the forking operation is shown in Fig. 3 for our FOREC algorithm.
As it can be seen, from the market-agnostic NMF model trained on us and de, we fork market-specific NMF models for each of the models, shown on the right and left sides of the figure.
Assuming the MLP part of the NMF model containing layers, and one additional layer for the NMF model, our objective with forking is to maximize the reusability of the general parameters. To this end, Raghu et al. [41] studies the similarity between an adapted model's layers and the general model and suggests that the main body of the network barely changes and all the adaptation happens in the head layers of the network. Inspired by this finding, we freeze layers up to layer of the MLP network (1 ≤ ≤ ), the only layer of the GMF network, as well as the user and item embeddings learned with each of the sub-networks. Given that freezing some part of the network limits the capacity of the network for learning market-specific parameters, we add new market-specific layers on top of the original tower-style feed-forward network right after the NMF layer to increase the network's capacity. We call these new layers as MarketHead layers. We believe that our forking operation provides a network for balancing between the general market and target market-specific parameters after the final fine-tuning. The and values are experimentally explored on a few pairs and fixed for every market-specific forking in our experiments. The forking operation for each target market is shown in line 12 of Algorithm 1. Further experimental details are given in Section 6.1.

Fine-Tuning: Final Training on the Target Market
Over the forking step, we obtain a new market-specific NMF network for the desired target market that the bottom part of the network is frozen for any update for providing generalized internal features, middle part initialized with the general market that could easily adapted, and the final part that randomly initialized and needs further training. One advantage of such a design is that it facilitates the maintenance of the entire network with the availability of new interactions on other market(s). Having the new market-specific model forked and initialized as described, we simply fine-tune the model using only the data from the target market. We keep the loss function the same for this part. However, one could easily change the loss function with this step to better adapt the market needs in the target market. The fine-tuning operation for each target market is shown in line 13 of Algorithm 1.

EXPERIMENTS 6.1 Experimental Setup
Dataset. We use XMarket dataset for our experimental evaluations. We specifically focus on Elec. category across 8 markets presented in Table 1. We prepare our data similar to single-market experimental setup in the literature [1-3, 42, 53]. For the ratings, we filtered items and users that there exist less than five transactions. We follow a long line of literature and use leave-one-out evaluation [7,15,17,18,24,25,30].
Compared methods. In order to show the effectiveness of our method, we employ the following models on each target market: • GMF, MLP, NMF: The Generalized Matrix Factorization (GMF), Multi-Layer Perceptron (MLP) and Neural Matrix Factorization (NMF) models from [17] trained using only the target market. • GMF++, MLP++, NMF++: The simplest way of leveraging the cross-market data is to train the model on the interactions inside both the source and target markets by sharing the item and user representations. We equally sample from both markets in the training phase-equal number of training interactions from each market is used. We experimentally observed that this training provides higher performance compared to simple concatenation of both markets. • DDTCDR: CDR and CMR have similarities as discussed in Sec. 2.
To test whether high performing CDR methods can be used to effectively solve the CMR problem, we include the algorithm proposed in [30], as one of the recent strong CDR methods in our comparison. As the assumption with CDR is that the set of users are shared across two domains, the original model connects the user features between a pair of MLP networks using an orthogonal transformation matrix. We adapt the model into CMR by connecting the item features between two market's MLP networks. We performed a similar modifications for the CoNet's network structure proposed by Hu et al. [18] for the CDR problem. We only report DDTCDR due to its consistent superiority compared to CoNet and the space limitations. • MAML: Meta-learning is widely used in the recommendation literature for variety of problem settings-see Section 2. Here, we adapt the learning paradigm to the CMR by employing the MAML framework providing model-agnostic solution for meta-learning. Our MAML training phase is described in Algorithm 1 lines 1-10.
Here, after the training phase, we perform a single pass withshots of sampled interactions from the validation split of the target market and fast adapt the pre-trained model parameters to the target market. We observed no further improvements with more number of passes on the adaptations. This baseline provides the sole importance of our MAML adaptation to the CMR problem. • NMF-FOREC: In order to show the impact of our MAML-based pre-training with the FOREC, we pre-train our market-agnostic  Hyper-parameters. For GMF, MLP and NMF we follow [17] and set all the network structure and the latent factor dimension as suggested, i.e. [16,64,32,16,8] with 8 as embedding dimensions.
For the optimizer, we use Adam [27] and select the learning rate and 2 -regularization coefficient hyper-parameters using the validation data of a subset of our markets. For the learning rate we considered {0.1, 0.05, 0.01, 0.005, 0.001, 0.0001} and selected 0.01 for MLP and NMF and 0.005 for the GMF model. For regularization we observed that 1 −7 is the best among our consideration set. We use 4 negative training samples for each user re-sampled with each epoch. ReLU is used as the activation function. For DDTCDR we use the same hyper-parameters provided in the original implementation of [30]. As the model uses a preset embeddings, we employ the GMF model for the initialization.  [16,32,16], and evaluated experimentally using the validation data on a subset of markets. We selected (c) as our model's market head layers for all experiments. In addition, we observed that setting higher 2 -regularization with the fine-tuning step helps the overall performance, especially with lower resourced markets-we set it to 0.001 for all fine-tuning steps.
Evaluation Metrics. We use Hit-Rate (HR) and nDCG as our evaluation metrics, commonly used in the literature. We report these metrics for a cut-off of 10. Similar to other works, we constructed the ground truth using the buying behavior by considering an item as relevant if the user gave a rating. In addition, we follow the literature and sample 99 negative items for each user in our evaluations.

Results and Discussion
We compare our FOREC model to several baseline techniques discussed in Sec. 6.1 in terms of recommendation performance. In theory, each target market can be paired with each auxiliary source market. However, for the 7 markets that we consider, all possible pairings of source and target markets leads to 49 different settings. For better readability and due to space limitation, we report our results in the following scenarios for every target market: • Best-Src: Each of the parallel markets are once considered as source and the model is evaluated using the nDCG@10 on the validation set. For each method, the source market with the highest improvement on the validation set is selected. As such, for a single target market, the best source market for different CMR methods may be different. • Ave-Src: In order to provide a rough indicator of the safer choice of the CMR method on each target market along with an overall insight on different source selections, we report the average performance of each model using different source markets. • Fix-Src: We report the results when a fixed base market is available (i.e. us market) and all CMR methods can only use that market to improve the target market recommendation. Table 3 summarizes the evaluation results with and without crossmarket data. All three aforementioned source selection scenarios are reported respectively in the Table. The best performing method for each target market and each source selection scenario is shown by bold fonts. We observe that FOREC is the winner in almost all target markets and across all the source selection scenarios, except for jp and uk in Best-Src scenario, and jp and in for Fix-Src. In addition, we observe that even our simple baselines are able to utilize to some degree the cross-market data provided and improve the target market's recommendation performance. This suggests the importance of cross-market training, even with a fixed base market, for better recommendation systems across different local markets. It also suggests the importance of the model and source market selection for deployment purposes.
When the source market is selected by the validation performance (i.e. Best-Src scenario), FOREC and MAML noticeably outperform other CMR methods. For uk, MAML is slightly better than FOREC, though the difference is not substantial. For jp, MAML has a higher nDCG, but FOREC is better in terms of HR, suggesting that neither can be picked as the winner. For the other 5 markets, FOREC outperforms MAML both in terms of nDCG and HR.
Looking at the results in Ave-Src scenario, FOREC outperforms other powerful CMR methods on average, meaning that given a source and target market, FOREC is a safer choice for CMR. This is further illustrated in Table 4 where we report for each target market the number of source markets for which each of FOREC or MAML is the winner along with the relative percentage improvement of FOREC compared to MAML. As it can be seen, over all 49 possible Comparing the Fix-Src results, suggests that even though us is not the best source market for all target markets in terms of performance boost (see Best-Src scenario), comparing with the single market results, it can be observed that us helps all of the tested target markets and the CMR comes to its highest performance when used with our FOREC model. In addition, we notice that NMF-FOREC is performing better in two markets compared to FOREC. We hypothesize that this might be due to the relative data size differences between the us and each of these markets (see Table 2), performing the pre-training using MAML provides a limited added value.
Comparing NMF-FOREC with FOREC, we observe that in most of the target markets, FOREC performs better. Based on the observations from Table 3, in most of the cases, MAML performs better than NMF++ and FOREC performs better than MAML. This shows the significance of the MAML-based pre-training as well as forking and fine-tuning, as proposed by our model. Fig. 4 compares these four models more deeply. In this figure, the nDCG@10 improvement of these CMR models over the NMF on single market are shown for different target markets. For each method in each target market, the distribution of nDCG@10 improvements based on different source markets is given as a box plot. This figure provides a better illustration of the trend between these four models described earlier.
Finally, we observe that the adopted DDTCDR, as one of the state-of-the-art CDR methods, does not perform well for CMR. In order to adopt a CDR method to CMR scenario, as described before, the users and items should be interchanged: the shared users in CDR are analogous to the shared items in CMR. This change of perspective looks natural at first sight, but it introduces some issues. Here we discuss the issue. In the item recommendation problem, a number of past interactions of a user with items are used to predict her future interactions. This means that the evaluation is based on the accuracy of the predicted items for users. In CDR, the users are shared across domains and the interacted items from the source domain add to the per-user information of the target domain. Differently, in CMR, the items are shared across markets  and the users are separate. Changing the perspective of users/items during training but having a fixed evaluation based on the per-user predictions is the issue of naïvely adopting a CDR to CMR. An interesting future direction would be to analyze the impact of such adoption for user recommendation in CMR scenarios.

Impact of Cross-Market Training on Different Users
Here we study the impact of different cross-market training approaches on user groups in terms of their training data size. Following the work of Liu et al. [32] we split the users based on the interactions into five groups. The users with the least number of interactions are named cold (average of 5 interactions), and the ones with the highest number of interactions are called warm (average of 13.3 interactions). We create five equally-sized user splits and report the average nDCG@10 for each group in Fig. 5a. This figure contains the performance of four models on these five user groups in the ca market: single market NMF, NMF++, MAML and FOREC.
Here, we used de as the source market. 7 We observe that MAML and FOREC almost uniformly improve the performance over the single market NMF in all five user groups. Comparing FOREC with MAML, again we see a consistent improvement in all five user groups with a slightly bigger gap toward the cold user group. This observation, together with other similar observations on our tests over other source-target pairs, provide experimental evidence that FOREC is suitable both for cold-and warm-start situations in the target market. NMF++, on the other hand, only helps the cold user groups, as can be seen in the figure.

Impact of the Target Market Data Size
In order to study the impact of target training data size, for a given market pair, we gradually decrease the number of training interactions for each user. Starting from the full target market user-item interactions, each time we halve the training data for the target market until only 10% of the target market's training interactions remains for each user. We train four of our models (NMF, NMF++, MAML, FOREC) on each of these settings and test on the target market. Fig. 5b presents the resulting nDCG@10 performance on ca with uk as the source market-similar observations made for a many pairs. As it can be seen, among the cross-market methods we see that FOREC and MAML are performing similarly especially 7 We repeated the experiments with different source and target markets and got similar results. Here we only show one case.
when the target market's size is extremely small. On the extremely small target market size (10% of the data), we see that single market NMF model as well as NMF++ are performing better. As more data becomes available, a significant boost among cross-market methods are observed, especially with FOREC-as opposed to NMF with lazy reaction to the new data availability. With further data availability, we see the full superiority of MAML as well as FOREC. This analysis suggests that some minimum amount of training data in the target market is essential for the cross-market models that we examined in order to be able to make use of the auxiliary source market. We hypothesize that when the target market provides limited amount of training data the pre-training through the MAML approach shifts parameters more toward the source market compared to the NMF++ training approach. However, the general observation from this analysis could be that FOREC and MAML are relatively resilient to the amount of the target training data. We note that this requires further analysis which is out of the scope of our study.

CONCLUSION & FUTURE WORK
We studied the problem of recommending relevant products to users in relatively resource-scarce markets by leveraging data from similar or richer-in-resource auxiliary markets. To this aim, we introduced a large-scale real-life dataset, named as XMarket, providing product information and reviews on 18 Amazon marketplaces featuring 52.5 million user-item interactions. We hypothesized and showed through extensive experiments on 7 target markets that data from one market can actually be used to improve the performance in another. Our model, named as FOREC, demonstrates robust effectiveness, consistently improving the performance on target markets compared to competitive baselines selected for our analysis. In particular, FOREC improves on average 24% and up to 50% in terms of nDCG@10, compared to the NMF baseline. Our analysis and experiments suggest specific future directions in this research area. We show that models that are designed for CDR are not necessarily suitable for the market adaptation problem setting. One interesting extension of our study could be designing models that are domain and market agnostic in the sense that they can consume the data across different markets and domains and leverage that for the improved recommendation on a target market's specific domain. In addition, we believe that data filtering or selection across markets could potentially be helpful for the models we discussed in our study. Moreover, using data augmentation techniques to generate synthetic ratings for target markets [6,48] could be a potential solution for the extreme low-resource markets, i.e. cold-start markets. We believe that many potentially interesting problems are yet to be explored in the CMR area.