Syndicated Bandits: A Framework for Auto Tuning Hyper-parameters in
  Contextual Bandit Algorithms
  The stochastic contextual bandit problem, which models the trade-off between
exploration and exploitation, has many real applications, including recommender
systems, online advertising and clinical trials. As many other machine learning
algorithms, contextual bandit algorithms often have one or more
hyper-parameters. As an example, in most optimal stochastic contextual bandit
algorithms, there is an unknown exploration parameter which controls the
trade-off between exploration and exploitation. A proper choice of the
hyper-parameters is essential for contextual bandit algorithms to perform well.
However, it is infeasible to use offline tuning methods to select
hyper-parameters in contextual bandit environment since there is no
pre-collected dataset and the decisions have to be made in real time. To tackle
this problem, we first propose a two-layer bandit structure for auto tuning the
exploration parameter and further generalize it to the Syndicated Bandits
framework which can learn multiple hyper-parameters dynamically in contextual
bandit environment. We derive the regret bounds of our proposed Syndicated
Bandits framework and show it can avoid its regret dependent exponentially in
the number of hyper-parameters to be tuned. Moreover, it achieves optimal
regret bounds under certain scenarios. Syndicated Bandits framework is general
enough to handle the tuning tasks in many popular contextual bandit algorithms,
such as LinUCB, LinTS, UCB-GLM, etc. Experiments on both synthetic and real
datasets validate the effectiveness of our proposed framework.
