Learning to Collaborate in Multi-Module Recommendation via Multi-Agent
  Reinforcement Learning without Communication
  With the rise of online e-commerce platforms, more and more customers prefer
to shop online. To sell more products, online platforms introduce various
modules to recommend items with different properties such as huge discounts. A
web page often consists of different independent modules. The ranking policies
of these modules are decided by different teams and optimized individually
without cooperation, which might result in competition between modules. Thus,
the global policy of the whole page could be sub-optimal. In this paper, we
propose a novel multi-agent cooperative reinforcement learning approach with
the restriction that different modules cannot communicate. Our contributions
are three-fold. Firstly, inspired by a solution concept in game theory named
correlated equilibrium, we design a signal network to promote cooperation of
all modules by generating signals (vectors) for different modules. Secondly, an
entropy-regularized version of the signal network is proposed to coordinate
agents' exploration of the optimal global policy. Furthermore, experiments
based on real-world e-commerce data demonstrate that our algorithm obtains
superior performance over baselines.
