Finite-Time Analysis of Kernelised Contextual Bandits
  We tackle the problem of online reward maximisation over a large finite set
of actions described by their contexts. We focus on the case when the number of
actions is too big to sample all of them even once. However we assume that we
have access to the similarities between actions' contexts and that the expected
reward is an arbitrary linear function of the contexts' images in the related
reproducing kernel Hilbert space (RKHS). We propose KernelUCB, a kernelised UCB
algorithm, and give a cumulative regret bound through a frequentist analysis.
For contextual bandits, the related algorithm GP-UCB turns out to be a special
case of our algorithm, and our finite-time analysis improves the regret bound
of GP-UCB for the agnostic case, both in the terms of the kernel-dependent
quantity and the RKHS norm of the reward function. Moreover, for the linear
kernel, our regret bound matches the lower bound for contextual linear bandits.
