Batch-Constrained Distributional Reinforcement Learning for
  Session-based Recommendation
  Most of the existing deep reinforcement learning (RL) approaches for
session-based recommendations either rely on costly online interactions with
real users, or rely on potentially biased rule-based or data-driven
user-behavior models for learning. In this work, we instead focus on learning
recommendation policies in the pure batch or offline setting, i.e. learning
policies solely from offline historical interaction logs or batch data
generated from an unknown and sub-optimal behavior policy, without further
access to data from the real-world or user-behavior models. We propose BCD4Rec:
Batch-Constrained Distributional RL for Session-based Recommendations. BCD4Rec
builds upon the recent advances in batch (offline) RL and distributional RL to
learn from offline logs while dealing with the intrinsically stochastic nature
of rewards from the users due to varied latent interest preferences
(environments). We demonstrate that BCD4Rec significantly improves upon the
behavior policy as well as strong RL and non-RL baselines in the batch setting
in terms of standard performance metrics like Click Through Rates or Buy Rates.
Other useful properties of BCD4Rec include: i. recommending items from the
correct latent categories indicating better value estimates despite large
action space (of the order of number of items), and ii. overcoming popularity
bias in clicked or bought items typically present in the offline logs.
