Generative Adversarial User Model for Reinforcement Learning Based
  Recommendation System
  There are great interests as well as many challenges in applying
reinforcement learning (RL) to recommendation systems. In this setting, an
online user is the environment; neither the reward function nor the environment
dynamics are clearly defined, making the application of RL challenging. In this
paper, we propose a novel model-based reinforcement learning framework for
recommendation systems, where we develop a generative adversarial network to
imitate user behavior dynamics and learn her reward function. Using this user
model as the simulation environment, we develop a novel Cascading DQN algorithm
to obtain a combinatorial recommendation policy which can handle a large number
of candidate items efficiently. In our experiments with real data, we show this
generative adversarial user model can better explain user behavior than
alternatives, and the RL policy based on this model can lead to a better
long-term reward for the user and higher click rate for the system.
