Learning List-wise Representation in Reinforcement Learning for Ads
  Allocation with Multiple Auxiliary Tasks
  With the recent prevalence of reinforcement learning (RL), there have been
tremendous interests in utilizing RL for ads allocation in recommendation
platforms (e.g., e-commerce and news feed sites). To achieve better allocation,
the input of recent RL-based ads allocation methods is upgraded from point-wise
single item to list-wise item arrangement. However, this also results in a
high-dimensional space of state-action pairs, making it difficult to learn
list-wise representations with good generalization ability. This further
hinders the exploration of RL agents and causes poor sample efficiency. To
address this problem, we propose a novel RL-based approach for ads allocation
which learns better list-wise representations by leveraging task-specific
signals on Meituan food delivery platform. Specifically, we propose three
different auxiliary tasks based on reconstruction, prediction, and contrastive
learning respectively according to prior domain knowledge on ads allocation. We
conduct extensive experiments on Meituan food delivery platform to evaluate the
effectiveness of the proposed auxiliary tasks. Both offline and online
experimental results show that the proposed method can learn better list-wise
representations and achieve higher revenue for the platform compared to the
state-of-the-art baselines.
