{'fraction_non_alphanumeric': 0.03037026546444476, 'fraction_numerical': 0.011738973075894192, 'mean_word_length': 4.831345200954304, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 24, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Fault in the Stars: Understanding Underground Incentivized Review Services

Product reviews play an important role in rankings and impact customers' purchasing decisions on e-commerce sites. There exists a thriving ecosystem of incentivized reviews on e-commerce marketplaces -- reviews written by real customers in exchange for free products. While some e-commerce marketplaces themselves support incentivized review programs to solicit honest high-quality reviews, there are parallel underground services that sellers can use to commission fake positive reviews from real customers in exchange for free products. Despite anecdotal reports, our understanding of how these incentivized services operate and, crucially, how are they able to resist takedown efforts is lacking. In this paper, we conduct a quantitative and qualitative study of incentivized review services by infiltrating an underground incentivized review service geared towards Amazon.com. On a dataset of 1600 products seeking incentivized reviews, we first demonstrate the ineffectiveness of off-the-shelf fake review detection as well as Amazon's existing countermeasures. Through a survey of more than 70 participants of this underground incentivized review service, we uncover fairly sophisticated recruitment, execution, and reporting mechanisms they use to scale their operation while resisting takedown attempts.

is not new. Traditionally, fake reviews were simply opinion spam where either bots or crowdworkers [15]. These fake reviews exhibited certain distinguishing characteristics such as short or low quality reviews, bursty reviews of different products in a short period of time, etc. These signals have proven effective in detecting outright fake reviews.
Many sellers, especially with new products without many reviews yet, are willing to pay to obtain positive product reviews [24,35]. E-commerce sites prohibit sellers from soliciting reviews in exchange for free products or other monetary incentives [9]. Instead, they themselves provide incentivized review services to help new sellers gain critical reviews early on. For example, Amazon Vine [1,2] is an incentivized review service that enables sellers to send free items to an inviteonly group of reviewers that are expected to provide honest and unbiased reviews. Since not every seller gets access to these incentivized review programs and more importantly get guaranteed positive reviews, an underground ecosystem of incentivized reviews has emerged over the last few years.
These underground incentivized review services allow sellers to solicit guaranteed positive reviews from real customers in exchange for free products. Since this practice is prohibited on e-commerce platforms [9], buyers actually purchase the product on the e-commerce platform but then get reimbursed out-of-band (e.g., via PayPal) after they provide evidence of the submitted positive review. The underground incentivized review services mediate the interactions between buyers and sellers while receiving a commission from the sellers.
Incentivized reviews significantly differ from traditional fake reviews in several key ways. First, the incentivized review are not written by bots [26] or crowdworkers [14,42]. These reviewers are in fact actual customers of the e-commerce platforms with a seemingly benign history of making routine purchases. As a result, the incentivized reviews do not exhibit any obvious signs of automated behavior (as they are actually not automated). Furthermore, the account activity as a whole is also not indicative of fraudulent because part of the purchases may be genuine and non-incentivized. These crucial differences make it challenging to reliably detect the incentivized reviews. As evidenced by Amazon's string of lawsuits against the incentivized review services [3,12,13], shutting them down is an arduous and painstaking process.
Despite anecdotal reports about the existence and prevalence of such incentivized review services, prior work lacks an understanding of how these services operate and how are they able to resist takedown attempts. We conduct a quantitative and qualitative study of the incentivized review ecosystem by infiltrating one such underground incentivized review service that is geared towards Amazon.com. We first characterize 16 thousand products soliciting incentivized reviews by analyzing their meta-data on Amazon, including if and when Amazon removed any of the product reviews. We then survey more than 70 participants in this underground incentivized review service to uncover their recruitment, execution, and reporting mechanisms with an emphasis on their evasion tactics to resist takeout attempts.
We summarize our key contributions as follows: • We infiltrate an underground incentivized review service to gather a dataset of 3200 products and 400k corresponding reviews. Since this list of products was shared to solicit incentivized reviews, it provides us one-of-akind ground truth labels of incentivized reviews.
• We analyze whether these products soliciting incentivized products can be distinguished from a control set of products using features that are traditionally used for fake review detection. We show that while there are statistically significant differences, they are insufficient to build high precision classifiers.
• Our audit of incentivized reviews and their (lack of) removal shows that Amazon's existing countermeasures are not effective. Amazon was able to delist only 0.1% of the products that solicited incentivized reviews and new reviews are promptly added to make up for any removed reviews.
• Through a survey of agents (N = 38) and buyers (N = 36), we study the modi operandi of the incentivized review ecosystem shedding light on sophisticated recruitment, execution, reporting and evasion tactics. For example, buyers tend to carefully spread out their incentivized reviews over time, simulate an organic search and write some negative reviews to evade detection. Agents are trained to manipulate certain keywords (such as replacing review by re**view) so that they are not flagged by social media platforms.
Paper Organization. The rest of the paper is organized as follows. In Section 2, we provide an overview of the review ecosystem, discuss prior research and novelty of our work. Section 3 describes our study design and data collection strategies. Then, in Section 4, we present the statistical tests, ma-chine learning experiments, and audits of Amazon's countermeasures. Section 5 describes the qualitative analysis of our surveys of buyers and agents in this incentivized review service with an emphasis on the evasion tactics used to resist takedown attempts by Amazon. We conclude in Section 6 with suggestions of potential countermeasures. There are three major players in the incentivized reviews ecosystem; sellers, agents, and jennies. Sellers. Third-party sellers on e-commerce platforms can manufacture their products by themselves and just sell them through the platform. Additionally, anyone can buy products in bulk from a wholesaler like AliExpress, and use another platform (like Amazon, Target or Walmart) for sales; poor quality products can be purchased at a low price and sold at a significant margin. Sellers stand to gain the most from positive reviews; more positive reviews usually means better ranking and the product shown to more potential buyers. Note that here we specifically refer to third-party vendors as sellers, and not the e-commerce platform which itself is a first-party seller (for example, Amazon acts as a platform for other sellers, but also sells its own line of products through the Amazon Basics brand). Our goal here is to study third-party sellers who attempt to manipulate reviews in order to sell poor quality products. Jennies. Buyers who will buy a product and leave a fivestar positive review for it are referred to as jennies by the agents. Note that jennies are regular, paying customers of the e-commerce platform; they are sources of revenue, engagement and advertisement impressions. A jenny is different than a traditional crowd-worker; while crowd-workers rely on crowd-sourced tasks for their livelihood and typically do hundreds of tasks a day, jennies write the occasional fake review in exchange for a free product. The persistence of engagement held by a jenny is much shorter than that one would expect of a crowd-worker. Jennies engage in review fraud to get free products, and some even make a profit by reselling the free products [35]. Agents. These are the middle-men contracted by sellers to identify potential buyers. Through Facebook groups, Telegram and Slack channels, Discord and targeted advertisements, these agents reach out to buyers and help the sellers obtain incentivized positive reviews. They instruct buyers (jennies) on how to make purchases and what evasion tactics to use to avoid detection. They also communicate information about orders, reviews, and refunds (which happen through a medium outside of the e-commerce platform, such as PayPal) from buyers to sellers and vice-versa.

End-to-End Functioning
At a high level, a jenny leaves a five-star positive review for a product and gets the product for free in return. Figure 1 depicts how the incentivized reviews market functions end-toend. First, sellers identify potential buyers (also called jennies) via their agents (steps 1 -3). Agents assist jennies in searching for and buying the product (steps 4 -8). The jenny then sends a screenshot of their order, which is ultimately confirmed by the seller (steps 9 -11); the order screenshot contains an order number that the seller can use for tracking review and return activity on that order. After the jenny receives the product, they submit a review. All reviews need to be approved by Amazon; once a review is live, the buyer sends a screenshot to the agent which is ultimately confirmed by the seller (steps 13 -16). The seller now has a five-star positive review for their product; they pay the agreed-upon commission to agents and refund the buyer for the full price of the product (steps 17 -18), effectively making it free for them; they can either use it or resell it through eBay or Facebook Marketplace and make a profit.

Related Work
Prevalence of Fake Reviews. The issue of fake reviews is not a new one. There has been a lot of attention from the research community to detect and mitigate the problem of fake reviews [4,14,17,21,23,26,27,30,32,[41][42][43]46]. Prior works have detected the existence and impact of fake reviews on websites like TripAdvisor, Expedia [32]. Hu et al. [23] conducted a large-scale analysis on Amazon and Barnes&Noble and found that consumers cannot fully isolate incentivized reviews from genuine reviews and can be easily misled. However, their investigation relies on heuristic signals of a product having fake reviews and not absolute ground truth.
Detection of Fake Reviews. Fake reviews have evolved over time in terms of the strategy and mechanisms through which they are generated. Traditional fake reviews were generated and posted by automated bots; detection methods for these rely on exploiting features commonly seen in bots such as high burstiness or low entropy in text [16,26,40]. Later research focused on reviews generated by crowdworkers. Kaghazgaran et al. [27] and Fayazi et al. [14] studied unverified reviews 1 written by crowdworkers, and compared it with legitimate reviews. Fornaciari et al. [17], generated a set of fake book reviews, and use natural language processing and machine learning classifiers to detect them; their accuracy, however, is a little over 75%. Luca and Zervas [30] explored the same problems that we do but for reviews on Yelp. Mukherjee et al. [34] presented a model for detecting fake reviews using purely the n-grams extracted from the review text, and obtained an accuracy of 68% which is low, but still indicates that the n-grams convey some signals. Barbado et al. [6] scraped user-centric features such as user activity across the platform and other behavior for fake review detection. Similarly, Wang et al.
[47] developed a detection approach that groups similar user clickstreams into behavioral clusters to identify groups of fake users, an approach that can potentially be used for detecting review fraud via sybil accounts.
Incentivized Fake Reviews. Incentivized reviews are becoming a popular marketing strategy used to solve the cold start problem and encourage consumers to buy and use a product [31,41]. Analogous to free product samples, the goal here is to promote the product by creating a 'buzz' [39]. Incentivized review services can help bootstrap a product and acquire some reviews when the product is new. Amazon Vine [1,2] is one such program where sellers can list their product to be shown to certain highly ranked reviewers (chosen by Amazon) who can then receive it for free and leave a review. Our focus, however, is not such official platform-run programs; instead, we study illegitimate incentivized reviews that are obtained by sellers without any disclosure and outside the e-commerce platform's knowledge. These underground marketplaces provide sellers a distinct advantage over the official programs: reviews are guaranteed within a short timespan. Most importantly, they are guaranteed to be five-star. Fake review marketplaces, especially for Amazon, have received wide media coverage and reports [11,29,36] reveal that Amazon is trying to crack down on these incentivized reviews using machine learning systems in combination with social media to detect them and stop them at source. The work that is most closely aligned with ours is a 2018 investigation by Nguyen [35]. Through interviews with a few complacent buyers and analysis of social media channels for review brokering, they investigate the fake reviews market and incentives of buyers and the operation of incentivized review services through Facebook groups, Slack channels and Reddit threads. Novel focus of our work. Our work is distinct from prior work in a number of aspects. First, unlike prior work, the reviewers we consider are not bots [26] or crowdworkers [14,42]. Our reviewers are real amazon customers who buy regularly from amazon, and a part of their purchases is genuine and non-incentivized as well. As a result, their activity does not exhibit typical 'fake' review characteristics like burstiness and high inter-review similarity. Second, unlike previous work, our labels are not based on heuristic signals [23]; we obtain and use a gold-standard dataset where we have a high confidence in our labels. Finally, we conduct qualitative investigations into the reviews market to identify key evasion tactics employed to make the incentivized reviews seem genuine.
To the best of our knowledge, ours is the first work that conducts an all-rounded analysis using gold standard datasets and qualitative investigation of underground incentivized review services. Additionally, while previous work has explored other underground markets [46] (such as those targeted towards Twitter, Gmail, Hotmail, YouTube), our work is the first to systematically study the underground marketplaces for incentivized reviews. While journalist investigations [35] apply a similar methodology to ours, they do not have systemic qualitative assessments at scale and fail to consider purposeful manipulation through evasion tactics. Additionally, they do not investigate the role of agents, the sophistication of the marketplace, and the various kinds of fraud within.

Data Collection
In this work, we study the functioning of the fraudulent reviews ecosystem. Being the largest e-commerce platform in the US, we use Amazon as a case study. This section describes how we crawl product and review meta-data from Amazon, and how we survey agents and jennies.

Data Crawling
Incentivized Products. We identified a few groups on Facebook where agents offer products for review (more information about groups in Section 5.3). We posed as buyers from the United States and posted in the group asking for any products that need 5-star review in exchange for a full refund. 8 agents responded to our post and provided us a link to a spreadsheet that listed the current product offerings. We obtained a list of around 1600 such products. We then systematically crawled product metadata (e.g., price, seller, variants) as well as reviews for each product including the review text, posted date, additional media attachments, and helpful votes. Because this list of products was obtained directly through agents, we have high confidence that the reviews of these products have been manipulated through incentivized review services. Control Products. Third-party sellers stand to benefit from incentivized reviews. Because of Amazon's prohibition of incentivized reviews, we argue that items shipped and sold by Amazon.com as the seller will be free from any review manipulation. We assume that all reviews for such products will be legitimate; as Amazon itself controls product rankings and appearance in search results, they have no reason to seek incentivized reviews. As a control group of products, we identified 1600 products that fit the criterion of Amazon.com being the seller and crawled their reviews. For an applesto-apples comparison, we chose products that were broadly similar to products in our list of incentivized products. Auxiliary Data. As described above, eight agents provided us with links to a spreadsheet with their list of products. Six of the agents shared links to sheets that contained multiple tabs, and information other than product details. These additional tabs contained links to other folders and documents as well. We collected all of this information manually and downloaded all files we could find. This supplementary material we crawled includes: (i) slide decks and video tutorials on how agents should communicate with jennies and form a network of jennies, (ii) what instructions are to be followed so that Amazon does not delete the review, and (iii) agent leader boards containing information about how many reviews each agent obtained and the bonus they earned. In our qualitative analysis, we draw on these sources as materials complementary to our survey responses.

Survey Protocol
In order to closely understand the inner-working of the incentivized review ecosystem, and examine the motivations and key tactics involved, we conducted semi-structured (consisting of both multiple-choice and free form response questions) surveys with agents and jennies. Participant Recruitment. We recruited participants for our study using Facebook groups where reviews are brokered by agents. We used the search term "Free Amazon Products" and ordered the search results by number of group members and post frequency. We joined the top ranking group (900k members and 38 posts per day on average) as it would provide a large sample to study the fake reviews market. By observing the posts and group activity, we identified which group members were agents (members who posted pictures of products and asked for reviews) and which of them were jennies (members who expressed their interests in receiving those products through comments on the posts). For every potential participant, we examined group activity over the past two weeks, and selected those which were fairly active (at least 10 posts or comments on 10 unique posts in the two-week period). Following this criteria, we identified the top 100 agents and jennies, and reached out to these individuals via Facebook Messenger. Potential participants were provided the details of our study (goals, nature of study, sample questions) and assured that we would not collect any PII (name, email, IP address). Once they consented, we sent them a link to our survey. All participants were over 21 years of age (self-reported), and were able to understand English. We surveyed a total of N = 38 agents and N = 36 jennies.
Methodology. Our surveys were conducted online via Google Forms. We posed questions in four main areas: operations (how the process works, key players), demographics (age, location, gender identity), incentives (motivations and earnings), and experience and evasion (tactics to avoid detection, observations, mental models). All of the questions were in English and completely optional. The questions and number of responses we received in both our groups can be seen in Appendix A. Note that the questions as presented in the table are in a generic form meant to describe the essence of the information being captured; in the actual surveys, we tailored each question for the specific group (agents or jennies) and added more descriptive explanations wherever needed.
Ethical Considerations. Two important ethical considerations arise as the result of this research. The first one pertains to ethics on handling human subjects research. We did not collect any personally identifying information during the course of this study. All the participants were aware that the answers they provide will be reported as part of a research study. Participants were recruited only after they provided us consent. All participants had the choice to decline to answer any of the questions, or withdraw their participation at any time during the survey. We treated all participants and their responses with respect, and followed principles laid down by the Menlo Report [28]. The second consideration pertains to ethics around revealing publicly the brands engaging in potentially illegal activity on Amazon. Throughout the course of our study, we identified several sellers and brands which engage in buying reviews for their products sold on Amazon. We do not reveal who these sellers are; if we were to do so, the sellers could trace the source back to our participants, which would jeopardize their anonymity. However, in order to further research in detecting incentivized reviews, we will share de-identified datasets, interview transcripts, key operational features and evasion tactics. Using our released artifacts, organizations will be able to follow the steps that we did to identify and infiltrate these marketplaces on social media and other platforms. Our recruitment strategy, surveys, and overall study protocol was reviewed by the Institutional Review Board (IRB) at our institution and determined to not be human subjects research.

Quantitative Analysis of Review Content and Meta-data
In this section, we first characterize potentially distinguishing features of incentivized reviews. We then use these features to develop machine learning classifiers for detecting products that solicit incentivized reviews. Finally, we conduct a longitudinal analysis added/removed reviews to understand the arms race between Amazon and incentivized review services.

Feature Analysis
Feature Engineering. Prior work has employed features such as review burstiness and inter-review similarity to detect fake reviews [26,34]; however, it is unclear whether these incentivized reviews might exhibit similar characteristics. Since the reviews for incentivized products were not organic, they might still show characteristics that differ from the control products.
Features we considered were based on prior work on fake review detection such as rating [43], review length (in terms of characters, words and sentences), number of punctuation characters used [16], and review polarity and subjectivity [37].
We also have additional signals such as the number of photos/videos attached and helpful votes for the reviews, so we leverage them as well. We hypothesize that the distribution of these features would also differ across the two groups due to review manipulation. Therefore, we considered both the mean and standard deviation in each feature [16,45]; this leads us to a total of 18 features. Additionally, because review manipulation involves adding more five-star positive reviews, we expect to see more significant differences in these features if computed over only five-star reviews for each product [34]; this gives us 16 features (all features replicated except mean and standard deviation in rating). Finally, we consider two more features: the ratio of one-star to five-star reviews, and the ratio of 'bad' (1/2/3-star) to 'good' (4/5-star) reviews. We therefore construct a total of 36 features and examine them for statistical significance. Analysis Methodology. For every feature, we calculated the mean in both groups of products and then compared the means using an unpaired two-tailed t-test [20]. Before performing the t-test, we validated the assumption of homogeneity in data using Levene's Test [8] and normality of data using the Shapiro-Wilkes Test [18]. Both tests showed high p-values (> 0.4) indicating that neither homogeneity nor normality assumptions of the t-test were violated. Our null hypothesis (H 0 ) for every feature was that the mean value of the feature would be the same in both the incentivized and control groups. The alternative hypothesis (H a ) was that the means would be different. Our tests showed that all of the differences were statistically significant at the 99% confidence level (p < 0.01). While all of our features turned out to be statistically significant, we present a subset here in Figure 2. Because of the statistically significant differences, and the clear visual distinc-tions in the distribution between the two groups, we expect that these features would be useful to classify products with and without incentivized reviews.

Detecting Incentivized Products
Using the aforementioned statistically significant features, we first trained machine learning classifiers [16] to detect incentivized products. We have 3200 products in total in our dataset. We reserve 33% of the data for testing, and train on the remaining 67%. We sample randomly but uniformly from both classes to derive our training and testing data. We also verified our results using 6-fold cross-validation. From the results shown in Table 1, we see that the random forest [22] is our best performing model with an accuracy of 82.14%. This equates to an error of nearly 20% which we contend is unacceptable for deployment in production systems. Precision-Recall Tradeoff. We further study the tradeoff between precision and recall to see if there exists a viable operating point in the tradeoff curve. In the scenario when an e-commerce company like Amazon detects incentivized reviews, they typically do one of two things: (i) remove the product listing from the website, or (ii) ban flagged user accounts from contributing to reviews. Note that these users were actual Amazon account holders. Similarly, Amazon earned revenue because of the product sales (even if they were purely incentivized). As a result, both banning the product or the user would result in reduced revenue for the platform. Therefore, any detection of incentivized reviews must operate at a high precision; that is, when a product is flagged, the chances of it being a false positive should be extremely low. This will, of course, come at the cost of low recall, which means that there would be some products which have incentivized reviews and would go undetected.

Model
Discussion. With our classification approach, we are able to obtain at most 81.06% precision at the product level. This means that of the products flagged as incentivized, 18.94% are false positives.While this number of false positives may seem reasonable, we contend that it would translate to a very large absolute number of false positives when considered at the scale at which Amazon (or any other large e-commerce platform) operates. We can force the models to operate at a higher precision by adjusting the classification threshold but it will come at the cost of a very low recall and low true positive rate (see Figure 3). Following the importance of high precision discussed above, false positives are expensive, and a system that is in production typically has to have extremely high precision (e.g., > 99%). Since we are unable to achieve low false positive rate, we conclude that detecting incentivized reviews is a challenging classification problem.

Auditing Incentivized Review Detection by Amazon
It is important to note that the precision we obtained on our machine learning models is likely a lower bound for Amazon's detection of incentivized products in the wild. Amazon has access to a variety of other signals such as user purchase history, search and navigation patterns, information on IP addresses and product sales trends, which we are unable to scrape as it is not public information. Therefore, we expect that Amazon will likely have more reliable algorithms to detect incentivized reviews. Review Deletion. In this section, we evaluate how effective Amazon is at detecting incentivized review manipulation. To this end, we periodically scrape product reviews for all products in our incentivized products dataset every week over a 6-week period. This allows us to examine how many reviews were added and deleted every week. While we cannot evaluate how precise their removal is (since we do not have access to all reviews they removed), we can evaluate the recall on our dataset of incentivized products. Figure 4 shows how reviews are deleted over the 6-week period. We consider all of the reviews in our original crawl as a baseline, and compute the proportion of reviews removed relative to this baseline. We then plot an empirical cumulative distribution function over products for every week. We can see that nearly 50% of the products seeking incentivized reviews have none of their reviews removed. Review Addition. Our analysis so far has considered how reviews are removed week-over-week. However, every week that the product is not deleted/delisted, the seller can acquire more incentivized reviews. When Amazon removes reviews, the seller can counter it by launching an incentivized review campaign to seek out more incentivized reviews. Figure 5 shows the percentage of reviews added and deleted each week relative to Week 1. This means that although we see some amount of reviews from Week 1 deleted till Week 6, there may be more reviews added in the intermediate weeks.
Week-over-Week Trends. Upon examining the week-overweek patterns for each product, we identify five main classes of products based on their review addition and removal trends: • Products which had some reviews removed, and then  launched an incentivized review campaign to recover from deletion (Figure 6a). We found 32 products displaying such behavior.
• Products which had some reviews removed, and did not attempt to gain more reviews (Figure 6b). We found 26 • Products which obtained reviews via a campaign, and these reviews were not removed by Amazon; the campaign was successful. (Figure 6c). We found 24 products displaying such behavior.
• Products which have ongoing campaigns; they are engaged with a cat-and-mouse game with Amazon. There are frequent review deletions, followed by review additions to recover from them (Figure 6d). We found 29 products displaying such behavior.
• Products which are eventually removed from Amazon. Figure 6e shows an example where a large number of reviews were removed in Week 2, and the seller tried to gain them back with an aggressive campaign in Week 3; the product was deleted in Week 4. We found 20 products displaying such behavior.
Discussion. As discussed in prior work [11], Amazon's antifraud systems would attempt to detect incentivized reviews, and routinely removes the detected reviews, products and user accounts from the platform. Our week-over-week analysis provides some evidence of review deletion by Amazon. However, out of 1600 products, Amazon was able to detect and eventually remove only 20 products from the platform. This shows that mitigating the problem of incentivized reviews is a challenging problem. Simply removing a small subset of the reviews does not solve the issue, as sellers can counter the removal simply by seeking more incentivized reviews.

Key Takeaways
Our quantitative analysis showed that there exist statistically significant differences in distributions between the incentivized and control products groups. For example, we observed that the reviews for incentivized products show a lower variance in features than control products. Features for control products show a higher variance, indicating natural variety in human opinions.These features, however, are not useful: machine learning models are able to achieve a precision of at most 82%, which translates to a non-trivial number of false positives in practice. Finally, we discovered that while Amazon is able to remove some incentivized reviews, their detection is fairly conservative. In response to the deleted reviews, sellers often onboard more incentivized reviews to account for the removal, leading to a cat-and-mouse game between Amazon and the sellers that seems to favor the latter.

Qualitative Analysis of Survey Data
To further understand the ineffectiveness of off-the-shelf machine learning classifiers as well as Amazon's existing countermeasures, we conduct semi-structured surveys with agents and buyers of the incentivized review service. Our thesis is that the challenges in detection of incentivized reviews likely stem from purposeful manipulation of user behavior on the platform.

Methodology
We now analyze the data we collected through our surveys of agents and jennies. The surveys were semi-structured, meaning that some responses were free-form. In order to identify key themes in our free-text survey responses, we used iterative coding inspired by grounded theory [10]. We analyzed our responses in batches of 5, and identified codes for every free-text response. In every batch, we recorded earlier batches if new codes had been found. We identified codes in broadly three phases of the incentivized reviews process: recruitment phase, purchase phase and review phase. The code book can be found in the Appendix C.

Demographics
Sellers. All of the agents we surveyed reported that the sellers they work with are based in China. The focus of this work is incentivized reviews on Amazon; however the underground reviews economy spans other platforms like Walmart (A8, A12, A35, J36), Wayfair (A20, J1, J19, J33) and Target (A3, A25, J14, J32). 9 agents reported that a seller often lists the same product on multiple e-commerce platforms.
Agents. The agents we surveyed are based in Pakistan (54%) and Bangladesh (46%). This is expected, as crowdworkers engaged in online fraud mostly come from such low-income countries [33]. Agents are 21 − 35 years old (µ = 25.6). Agents work in groups and each group has an admin. The admin is the interface between agents and the seller. An admin and his group of agents works for multiple sellers at the same time. The team size ranged from 10 − 200 members. Agents tend to have limited education; 20 agents had earned a diploma and 19 agents had only a high-school level education. Only 5 agents were pursuing further education (a diploma or bachelors degree).
Jennies. Jennies we surveyed were Amazon account holder based in the US, UK, and Canada. However, note that the agents reported that they also target jennies in Ireland, France, and Germany as well since some products are geared towards those countries. Jennies were 22 − 51 years old (µ = 31.6). All of the jennies held at least a bachelors degree, with 15 holding a masters degree and 5 holding a doctorate. This level of education is significantly higher from that observed among fraudulent actors in prior work [5,38]. With their unusually high educational qualifications, it is likely that the jennies intricately understand the nature of fraud they are perpetrating, the safeguards that might be in place to detect it, and purposely manipulate their activities to evade them.

Recruiting Jennies
Agents are responsible for recruiting jennies and help them buy products. There are two mediums by which agents recruit reviewers: social media channels and targeted advertisements. Facebook Groups. We were able to find over 100 groups on Facebook which served as a hub for reviewers and sellers to contact each other. The largest of these groups had close to two hundred thousand members. After a recent Amazon lawsuit, most of these have now been taken down by Facebook. On these groups, agents post information about the products which they have. Reviewers can also ask for any products they want. Agents are aware that what their activities are potentially illegal, and hence, always operate with an alias. They also use minor perturbations on the keywords so that they may not be flagged by Facebook (see Fig. 7 in Appendix B). We discovered extensive training material complete with the scripts and scenarios on how agents can approach a jenny and walk them through the process. Jennies were part of at least 1 and at most 25 groups (µ = 6) on Facebook which advertised free products in exchange for five-star positive reviews.
The goal of the groups is to function as an exchange where reviews are bought and sold (J1, J11, J13, J15). Typically, agents post products on the group along with what the seller is looking for; it can be a five-star review, a five-star rating, positive seller feedback, upvotes to existing reviews or answering questions in the Q&A section (J24, J25, J28, J30, J33). Some jennies also reported that both jennies and agents post about scammers (more about the scams in Appendix D ) and warn others not to trust them (J4, J9, J21, J26). Targeted Advertisements. Most jennies (75%) reported that they were introduced to the incentivized reviews economy via targeted advertisements on Facebook. Facebook facilitates the delivery of advertisements to a targeted audience; therefore agents are able to leverage search history, marketplace activity and other data available to Facebook to identify users who might be interested in writing such incentivized reviews. Further, a direct fallout of the targeted advertisements is the rabbit hole phenomenon; once a user clicks or interacts positively with an advertisement for free products, he goes down the rabbit hole and sees several more advertisements of similar pages. As J1 says: "I just chanced upon this side-hustle one day; I clicked on the advertisement, and then saw a few more. Pretty soon, my news feed was full of such advertisements; every third or fourth post I saw was for free products in exchange for reviews." As Instagram is also owned by Facebook, jennies reported that once they interacted with a few advertisements on Facebook, they started getting exposed to similar ads (advertising free products in exchange for reviews) on Instagram as well (J13, J16, J22). An example of this is shown in Figure 8 in the Appendix B.

Agent Incentives
Incentives for agents are mainly monetary. Agents we surveyed are located primarily in low-income countries. Agents earn either $4 or $5 as commission on a review they helped procure, with the average being $4.42. The monthly income of agents (apart from the reviews) ranges from $0 to $300 (µ = $100.25). Most agents earn commission comparable to their monthly income; income from reviews ranges from 33% to 100% of their total income. Top agents in a group (those who helped obtain the maximum number of reviews) are also paid an additional bonus (A13, A19, A34). 7 of the agents have no other full-time job; the reviews are their only source of income. All of the agents were aware that by facilitating incentivized reviews, they are violating Amazon's terms of service. There are three important factors that contribute to agents' motivation: additional income that is comparable to full-time salary (15 agents), lack of requirement of any infrastructure (9 agents), and lack of special education or skills required (11 agents). According to agent A6, based in Bangladesh: "One month I earned almost $250; my regular job pays me around $100 a month. And it requires no additional expense; just an internet connection." Additionally, through the proxy materials, we discovered that there is an incentive structure to encourage agents to seek more reviews. Leader boards updated in real-time list agent names and number of reviews they helped procure. The top 3 − −5 agents at the end of every month receive additional commission.

Jenny Incentives
Major motivations for jennies were the opportunity to get products for free, and not having to pay for some expensive products (such as robot vacuum cleaners (J7), treadmill (J14) and blenders (J17)). J8 says: "I know it's wrong....but when I moved into my new place, I was able to get a chair, desk, humidifier, kitchen utensils, storage boxes, vacuum cleaners and a lot more for free. I saved so much by just writing reviews." Other buyers go a step ahead. They get their products for free by writing reviews, and then sell them online. Jennies reported selling products on sites like eBay, Facebook Marketplace and Offerup. Jenny J2 tells us: "The product is basically free. Once I get my refund, I sell it on Facebook marketplace for around 75% of the price. I made over $300 last month." When asked if they were aware that writing incentivized reviews is against Amazon's terms of service, 32 jennies (> 86%) responded Yes. 4 responded as Maybe and only 1 said No. We also see that the financial incentives for jennies are weaker than they are for our agents.
Jenny incomes range from $25, 000 − $230, 000 annually. Our lowest-earning jenny who is a graduate student earns $25, 000 a year, and the amount they save by writing reviews ($1200 in a year) is less than 5% of their annual income. This is in stark contrast with agents where the money earned via reviews forms at least a third of their total income (and note that agents earn a smaller amount per review than jennies save!).

Evasion Tactics
In this section, we discuss the evasion tactics that jennies are asked to follow. Agents and sellers provide these guidelines during the purchase and before writing a review. The goal is to minimize the likelihood of the purchase and the following review being detected and subsequently deleted by Amazon.
By analyzing our free-text survey responses, we identified the following 11 key evasion tactics. Organic Search for Product. Agents always recommend buyers to search for the product with keywords, rather than provide a direct link or the brand name. Buyers are then asked to scroll through the products and find the correct item. According to agents, this is because Amazon can track that you landed on that product from a link, and can consider it to be suspicious. An agent (A3) notes: "It is for security of your account. Sellers don't allow us to share direct links with buyers, as Amazon will detect this, and may remove the review that you write." Similarly, buyers are advised never to search for a product with the brand name in the search term. As far as possible, agents never directly disclose the brand name. A jenny(J2) describes this: "They never show us the brand. In the image they send us, the brand name is blurred out. I try to find the correct product and send them a screenshot or link to confirm." The goal of this is to simulate an organic search by a genuine user. Platform Engagement. As an extension of above, jennies are encouraged to engage with the Amazon platform while searching for the product. They are asked to look at similar products, browse through images, and read reviews. The goal of this is to simulate an organic product discovery experience. An agent(A2) says: "We always ask buyers to spend at least 1 minute browsing similar products, and at least 15 seconds on checkout page since if search and purchase is done too quickly, it is suspicious to Amazon." Jennies generally follow the guidelines, since their accounts are at risk. A jenny (J18) also concurs, saying: "I spend a lot of time in searching for the products. I really read 5-10 5* and some 1* reviews. I also ask a question or two in the forum. For some of my trusted agents, I also write reviews AFTER the return period. That will tell Amazon that my product is not probably refunded (if it was, I wouldn't risk losing the 30 day return period)." Payment Restrictions. Jennies are advised not to pay using gift cards, and always make the full payment. In addition, Amazon at times displays coupons using which buyers can get a discount on the product; buyers are cautioned not to use these either. The reason behind this is that the seller has designated a fixed number of coupons, and these are to attract other, genuine customers to buy the product. An agent (A7) says: Tactic n (Agents) n (Jennies) Search product with keywords 11 (28.94%) 10 (27.78%) Spend time reading features and reviews 7 (18.42%) 7 (19.44%) Mark reviews as helpful and ask questions in the QnA 12 (31.57%) 8(22.22%) Add similar products to saved lists and shopping cart 8 (21.05%) 9 (25%) Not paying with gift cards or coupons 6 (15.79%) 2 (5.5%) Wait 10 − 15 days to submit a review 11 (28.94%) 4 (11%) Add photos and videos to submitted review 13 (34.21%) 12 (33.33%) Write reviews of at least 300 words 8 (21.05%) 1 (2.25%) Avoid buying from the same seller 3 (7.89%) 2 (5.5%) Write some 1-star and 2-star reviews for other products 13 (34.21%) 11(30.55%) Be consistent across all reviews in terms of timing, content, media 0 (0%) 6(16.66%) Table 2: Key Evasion Tactics employed by Jennies and recommended by Agents to avoid detection of reviews by Amazon.
"Most sellers tell us not to ask buyers to use coupons. If they do, they still refund them, as it is not a loss. But the coupons are mainly there for other customers (who we will not be refunding) to think that they are getting a discount." Our hypothesis behind sellers not allowing the use of a gift card is that since a gift card works via a coupon code, it could be construed by Amazon as a discount offered by the seller. Jenny (J7) notes: "....I generally try not to use the gift card balance. One time I accidentally used $5 from my gift card balance, and they refunded me $5 less. Now I have to manually uncheck any gift card balance available, since Amazon uses it by default." Review Timing. Jennies are recommended to wait at least 10 days before submitting a review on Amazon. Reviews which are submitted too soon arouse suspicious. According an agent (A3): "...Please submit reviews 10-15 days after you receive the shipment. Use the product for a few days and then review.
Else, your review will be removed by Amazon." Buyers are, of course, eager to receive the refund as soon as possible. However, they understand the reasoning. Jenny JJ says: "It makes sense. I need to have experienced the product before writing a review, otherwise it is suspicious. I maintain a spreadsheet of items I purchased along with their dates, so I know when 10 days have passed to write the review." Review Content. Agents provide detailed instructions to buyers on the content of the reviews. According to agents, factors like the length of the review, and presence of supplementary media (images and videos) affects whether Amazon will remove the review. All ratings have to be five-star. Agents always recommend writing a review of at least 300 words. A jenny (J6) tells us: "Picture and video reviews are really important to them. Once, an agent offered me additional commission of $5 if I wrote a review of more than 200 words. And $5 more if I attached a picture or video to it." However, 5 jennies reported that they do not follow this advice because they find it challenging to come up with a long, positive, five-star fake review (J1, J8, J12, J25). In addition to review length, they are cautioned never to strongly emphasize any particular seller. An agent (A11) notes: "Never ever mention 'always buy from this seller' in your review. We want the product from our seller to be highly ranked. If you write about a seller, Amazon moves the review to seller feedback, which sellers don't want, because customers don't view seller feedback often." Seller Diversity. In general, buyers are not permitted to review multiple products from the same seller within a certain period of time. According to agent A8: "Very suspicious if you buy too many products from same store. I never return to the same buyer before 2 months of the first review." Buyers have another thumb rule which they follow, and that is avoiding buying too many products offered by the same agent. Jenny J10 shares their experience: "I purchased a vacuum cleaner and sent the order screenshot to the agent. She confirmed my order, and showed me some more products to buy. When I went to amazon, I saw the SAME product as a suggestion, under the heading 'people with similar purchases to you also bought....'. It would have been very suspicious. Clearly, a pattern had been captured." Account Meta-data. Agents and sellers also take into consideration account meta-data while analyzing whether a particular buyer's review is likely to be removed or not. Of our 39 participating agents, 31 reported that they strongly prefer Amazon prime account holders. According to agents, reviews from accounts who have Amazon Prime subscriptions are less likely to get removed. 24 of our participating agents reported that when some products had some reviews removed, all of them were from non-Prime accounts. Agent A9 says: "Amazon moderates all reviews before they go live on the product page. Reviews by prime accounts are approved in around 36 hours, but it takes around 96 hours (4 days) for reviews from non-prime accounts. " In addition, agents also check if a prospective buyer is an easy rater, that is, he has too many five-star reviews and no three or four-star reviews. Agents also recommend that reviewers write reviews for items that they have purchased organically (that is, without rebate). They also recommend that not all reviews be five-star. Jennies are aware that a diversity in ratings is important to keep up the impression that reviews are genuine. Jenny J16 says: "I have some non-5* reviews on my profile. If I write only 5*, amazon will think I am writing fake reviews, if I add some 1,2* they will think I am doing it to balance out, but I write some 3/4*; there is no clear incentive for me, which increases their confidence that I am a genuine buyer." We would like to note here how this need for rating diversity can be harmful for other sellers. 11 jennies reported that for the products which they purchased by themselves, or were not incentivized, they left a negative review even if the product was good so that they would have some 1-star and 2-star reviews on their reviewer profile. Review Consistency. This strategy is unique because none of the agents expressed it as an evasion tactic, but 8 jennies shared that they kept all of their reviews consistent in terms of content, length, tone, and media attachments. Jenny J34 says: "I try to be consistent across all reviews so amazon doesn't feel I'm giving special attention to any product. All my reviews are same length, all have photos, etc." Having these attributes consistent over various products (both incentivized and genuine purchases) makes it harder for Amazon to detect the incentivized reviews as different. It is interesting to note that jennies are able to understand the significance of consistency and employ it as an adversarial tactic.
Discussion. Based on our survey responses, it appears that both agents and jennies have a non-trivial understanding of how incentivized fraud detection works, and what features Amazon might examine to detect it. Agents seem to know a lot about the internal workings of amazon's detection models (A5, A18, A33). An agent A5 tells us: "There are two computer programs that Amazon uses to detect reviews. One looks at content and the other looks at timing, search behavior, etc. We know that the review content program checks if you have shared your opinion and not simply information about the product." Two jennies (J11, J24) state that they work in the technology sector, understand machine learning modeling, theorize what features Amazon uses to flag reviews, and then purposely attempt to circumvent detection by manipulating those signals. J11 tells us: "I work in tech so I am aware that machine learning models can catch suspicious activity, so I try to simulate a real search as much as possible. I browse several products, read reviews and mark as helpful. Sometimes I keep things in my cart for days. I also randomly search for products and not buy anything. Also, for some products I ordered genuinely (no refund) I write 1* reviews even if they are good, so I am not an 'easy' reviewer. " This supports our hypothesis of purposeful and deliberate manipulation that jennies engage in to perpetrate this fraud and remain undetected.

Key Takeaways
Our survey shows that the incentivized reviews economy spans across multiple countries, with sellers in China, agents in Pakistan and Bangladesh, and jennies in the US, UK, and Canada. The network of agents is fairly sophisticated and uses analytical dashboards and incentive schemes. Demographic analysis shows that while agents come from low-income countries and have moderate educational qualifications, jennies are highly educated (some holding doctoral degrees) and at a high income level. For agents, the motivations to perpetrate fraud are purely monetary, given that for some of them, commissions earned on incentivized reviews are the only source of income. Jennies are recruited through targeted advertisements on Facebook and Instagram. Private Facebook groups serve as platforms where reviews and ratings are brokered. Jennies use various evasion tactics (such as simulating an organic search, including media attachments) to deliberately manipulate their account activity and evade detection by Amazon.

Concluding Remarks
In this paper, we studied the underground marketplace of incentivized reviews and ratings. By infiltration an incentivized review service, we were able to compile a list of products that solicit incentivized reviews on Amazon. Our quantitative analysis of these products showed that while they do have some distinguishing characteristics, these are insufficient to build reliable machine learning models that can automatically detect incentivized reviews without too many false positives. Our audit of Amazon's existing countermeasures by way of incentivized review deletion showed that these services are either able to evade detection or are able to keep adding new incentivized reviews if Amazon does delete some of them. The qualitative analysis of our survey of agents and jennies identified various tactics employed by agents and jennies to evade detection by Amazon. For example, jennies sometimes report writing false negative reviews for genuine products to keep a balanced review profile. In summary, we find that the problem of detecting incentivized reviews is a challenging one, particularly because: (i) these incentivized reviews are from real people that leave no obvious trace of the fraudulent activity for Amazon to detect; and (ii) Amazon likely has to be conservative in its countermeasures to avoid false positives and resulting revenue/user loss. Below, we describe some promising existing and new ideas to mitigate the problem of incentivized reviews on ecommerce sites.
While our work points out shortcomings in Amazon's existing countermeasures, we do not claim that Amazon is not seriously attempting to address the issue of incentivized reviews on their platform. For instance, it is likely that Amazon does detect a significant number of incentivized reviews before that are ever posted on product pages. These cases of successful countermeasures are invisible to our external audit. We cannot assess these countermeasures without engaging in fraudulent activity ourselves (e.g., posting incentivized reviews as jennies and seeing whether Amazon removes the reviews or disables the account).
The public disclosures show that Amazon seems to take the issue of incentivized reviews seriously and has recently cracked down on users who post fraudulent reviews and sellers who seek them. For example, Amazon conducted a sting operation [12] where their undercover investigators posed as sellers and recruited jennies using Fiverr 2 ). Similarly, Amazon recently took legal action [3] against Rebatest 3 , App-Sally 4 and AMZTigers [13]; all of which function as marketplaces for incentivized reviews.
Building on these countermeasures, we believe that ecommerce platforms can attempt to employ honeypot products and seek incentivized reviews from incentivized review services. To identify fraudulent products and sellers, it could be in the form of posing as a buyer willing to write a five-star review in exchange for a refund, or an agency claiming to provide five-star reviews for cheap. To identify user end, it could be in the form of posing as an agent and offering a refund for a five-star review. While this approach may not completely eliminate the issue of incentivized reviews longterm, it can definitely be used as a confirmatory test to validate high-confidence detection and reduce false positives in the short-term.
Through our surveys, we also identified several previously unknown tactics used to evade detection. Inspired by these, we can engineer additional features to detect incentivized reviews at a user level. For example, a reasonable consumer will try to get the best price for a particular product he is buying. However, as discussed earlier, jennies are instructed to not use any coupons offered by Amazon. Similarly, if there is any existing gift card balance in an account, Amazon automatically applies that to any purchase you make. In order to not use that balance, the user needs to select the payment methods menu, and manually uncheck the box corresponding to the gift card balance. If a buyer deliberately avoids obvious coupons and gift card balance on their account, it may indicate that the purchase and review is incentivized. Additionally, jennies use reminders and spreadsheets to keep track of when to write reviews (10-15 days after receiving the product). Monotonicity in time to review is another signal that could be leveraged. Finally, e-commerce platforms have access to a number of back-end features that we do not; e.g., product sales trends, information about how a customer searches and ends up buying a product, which users had written reviews for products confirmed as being fraudulent, IP addresses and other network traffic patterns of users/products, etc.
Agents have their network of jennies built via Facebook groups and prior purchases who they show products to and seek reviews. Because of this, there are likely groups of users that always review particular groups of products. A graph can be built with users and products as nodes with an edge if a user has reviewed a product. Off-the-shelf community detection or clustering algorithms may help us identify clusters in users and products [7,25]. Because of the natural graph structure, advanced graph machine learning (such as graph neural networks) could be used to detect malicious users, products, review activity, or fraudulent sub-graphs of users and products. This approach can be used in conjunction with other features to detect incentivized reviews, sellers/products that seek them, and users who write them. A Survey Questionnaire Table 3 lists the questions we posed to jennies and agents in our survey, and the response rate in each group.

B Jenny Recruitment
Recruitment of Jennies via Facebook groups with manipulated words can be seen in Figure 7. The power and role of targeted advertisements can be seen in Figure 8.  36 Have you ever been a victim of a scam while seeking or writing incentivized reviews?* 13 18 Table 3: Survey Questions posed to participants. N Agents and N Jennies denote the number of agents and jennies respectively who provided and answer to the question. Questions marked with * indicate that a free-form text response was expected. N/A indicates that the question was not posed or not applicable to that specific group.

D Fraud Within Fraud
Like any other ecosystem, the reviews ecosystem also has its share of malicious actors who attempt to game the system. All three key players (sellers, agents and jennies) have some motive for fraud. Based on our interviews, we found three common methods of fraud within the fraudulent reviews economy. Buyer Fraud. This is an instance of jennies scamming agents and sellers. 15 of the agents we surveyed reported that they had encountered at least one jenny who would order the product, leave a review, receive a refund, and then return the product via Amazon. The buyer receives a free product, and earns additional money (the full cost of the product is refunded by Amazon). This defeats the purpose of seeking incentivized reviews; sellers lose an amount double the cost of the product (one via the fraudulent refund, the other via Amazon), and a return of the product actually affects ratings and rankings negatively. Agents do not get paid their commission if the jenny behaves this way as it was the agent's responsibility to vet a jenny (A6, A10, A18).
Seller Fraud. This is an instance of sellers scamming agents and jennies. Some sellers engage agents solely for the purpose of driving down the competition. The seller poses as a competitor seller and lists competing products with the agents, typically asking for a five-star rating. After jennies buy the product, and leave a rating, the seller never refunds them. This can have two fallouts for the competitor. First, jennies who did not receive the refund will return the product (J8, J15, J16, J26). Second, these jennies will leave a negative review for the product, since they believe that the competitor seller scammed them (J3, J10, J11, J20, J26, J28). Both of these negatively affect ratings and statistics of competitor products, enabling the seller to drive down their rankings. A jenny J11 shares their anecdote: "Some agents purposely suggest competitor seller products and steal your refund so you write a -ve review for competitor seller. I found out once when agent did not refund so I emailed the seller and they said they don't have any agents." It is not clear whether agents are complicit with sellers in committing competitive review fraud. However, if they were, it would work to their disadvantage; they depend on jennies to obtain reviews and commissions on an ongoing basis. 4 jennies and 5 agents from our survey respondents reported that they had experienced such fraud at least once. Agent Fraud. This is an instance of agents scamming sellers and jennies. Agents obtain the order and review details from jennies, and submit it to the seller. However, while doing so, they use their own PayPal account and ask the seller to refund to that account (J8, J13, J17, J28). Thus, the jenny buys and reviews the product, but the agent earns the refund. Such agents are reported in the Facebook groups where products are listed (J21). 17 jennies reported that they had come across such agents at least once. Although agents are scamming jennies, this indirectly harms the seller as well; as described above, when jennies don't receive a refund, they leave negative reviews for the product.