Neural Contextual Bandits without Regret
  Contextual bandits are a rich model for sequential decision making given side
information, with important applications, e.g., in recommender systems. We
propose novel algorithms for contextual bandits harnessing neural networks to
approximate the unknown reward function. We resolve the open problem of proving
sublinear regret bounds in this setting for general context sequences,
considering both fully-connected and convolutional networks. To this end, we
first analyze NTK-UCB, a kernelized bandit optimization algorithm employing the
Neural Tangent Kernel (NTK), and bound its regret in terms of the NTK maximum
information gain $\gamma_T$, a complexity parameter capturing the difficulty of
learning. Our bounds on $\gamma_T$ for the NTK may be of independent interest.
We then introduce our neural network based algorithm NN-UCB, and show that its
regret closely tracks that of NTK-UCB. Under broad non-parametric assumptions
about the reward function, our approach converges to the optimal policy at a
$\tilde{\mathcal{O}}(T^{-1/2d})$ rate, where $d$ is the dimension of the
context.
