Learning to Advertise for Organic Traffic Maximization in E-Commerce
  Product Feeds
  Most e-commerce product feeds provide blended results of advertised products
and recommended products to consumers. The underlying advertising and
recommendation platforms share similar if not exactly the same set of candidate
products. Consumers' behaviors on the advertised results constitute part of the
recommendation model's training data and therefore can influence the
recommended results. We refer to this process as Leverage. Considering this
mechanism, we propose a novel perspective that advertisers can strategically
bid through the advertising platform to optimize their recommended organic
traffic. By analyzing the real-world data, we first explain the principles of
Leverage mechanism, i.e., the dynamic models of Leverage. Then we introduce a
novel Leverage optimization problem and formulate it with a Markov Decision
Process. To deal with the sample complexity challenge in model-free
reinforcement learning, we propose a novel Hybrid Training Leverage Bidding
(HTLB) algorithm which combines the real-world samples and the
emulator-generated samples to boost the learning speed and stability. Our
offline experiments as well as the results from the online deployment
demonstrate the superior performance of our approach.
