Recommendation System-based Upper Confidence Bound for Online
  Advertising
  In this paper, the method UCB-RS, which resorts to recommendation system (RS)
for enhancing the upper-confidence bound algorithm UCB, is presented. The
proposed method is used for dealing with non-stationary and large-state spaces
multi-armed bandit problems. The proposed method has been targeted to the
problem of the product recommendation in the online advertising. Through
extensive testing with RecoGym, an OpenAI Gym-based reinforcement learning
environment for the product recommendation in online advertising, the proposed
method outperforms the widespread reinforcement learning schemes such as
$\epsilon$-Greedy, Upper Confidence (UCB1) and Exponential Weights for
Exploration and Exploitation (EXP3).
