An Empirical Study of Neural Kernel Bandits
  Neural bandits have enabled practitioners to operate efficiently on problems
with non-linear reward functions. While in general contextual bandits commonly
utilize Gaussian process (GP) predictive distributions for decision making, the
most successful neural variants use only the last layer parameters in the
derivation. Research on neural kernels (NK) has recently established a
correspondence between deep networks and GPs that take into account all the
parameters of a NN and can be trained more efficiently than most Bayesian NNs.
We propose to directly apply NK-induced distributions to guide an upper
confidence bound or Thompson sampling-based policy. We show that NK bandits
achieve state-of-the-art performance on highly non-linear structured data.
Furthermore, we analyze practical considerations such as training frequency and
model partitioning. We believe our work will help better understand the impact
of utilizing NKs in applied settings.
