RL4RS: A Real-World Benchmark for Reinforcement Learning based
  Recommender System
  Reinforcement learning based recommender systems (RL-based RS) aim at
learning a good policy from a batch of collected data, by casting sequential
recommendations to multi-step decision-making tasks. However, current RL-based
RS benchmarks commonly have a large reality gap, because they involve
artificial RL datasets or semi-simulated RS datasets, and the trained policy is
directly evaluated in the simulation environment. In real-world situations, not
all recommendation problems are suitable to be transformed into reinforcement
learning problems. Unlike previous academic RL research, RL-based RS suffers
from extrapolation error and the difficulties of being well-validated before
deployment. In this paper, we introduce the RL4RS (Reinforcement Learning for
Recommender Systems) benchmark - a new resource fully collected from industrial
applications to train and evaluate RL algorithms with special concerns on the
above issues. It contains two datasets, tuned simulation environments, related
advanced RL baselines, data understanding tools, and counterfactual policy
evaluation algorithms. The RL4RS suit can be found at
https://github.com/fuxiAIlab/RL4RS. In addition to the RL-based recommender
systems, we expect the resource to contribute to research in reinforcement
learning and neural combinatorial optimization.
