Contextual Combinatorial Volatile Bandits with Satisfying via Gaussian
  Processes
  In many real-world applications of combinatorial bandits such as content
caching, rewards must be maximized while satisfying minimum service
requirements. In addition, base arm availabilities vary over time, and actions
need to be adapted to the situation to maximize the rewards. We propose a new
bandit model called Contextual Combinatorial Volatile Bandits with Group
Thresholds to address these challenges. Our model subsumes combinatorial
bandits by considering super arms to be subsets of groups of base arms. We seek
to maximize super arm rewards while satisfying thresholds of all base arm
groups that constitute a super arm. To this end, we define a new notion of
regret that merges super arm reward maximization with group reward
satisfaction. To facilitate learning, we assume that the mean outcomes of base
arms are samples from a Gaussian Process indexed by the context set ${\cal X}$,
and the expected reward is Lipschitz continuous in expected base arm outcomes.
We propose an algorithm, called Thresholded Combinatorial Gaussian Process
Upper Confidence Bounds (TCGP-UCB), that balances between maximizing cumulative
reward and satisfying group reward thresholds and prove that it incurs
$\tilde{O}(K\sqrt{T\overline{\gamma}_{T}} )$ regret with high probability,
where $\overline{\gamma}_{T}$ is the maximum information gain associated with
the set of base arm contexts that appeared in the first $T$ rounds and $K$ is
the maximum super arm cardinality of any feasible action over all rounds. We
show in experiments that our algorithm accumulates a reward comparable with
that of the state-of-the-art combinatorial bandit algorithm while picking
actions whose groups satisfy their thresholds.
