Pure Exploration in Kernel and Neural Bandits
  We study pure exploration in bandits, where the dimension of the feature
representation can be much larger than the number of arms. To overcome the
curse of dimensionality, we propose to adaptively embed the feature
representation of each arm into a lower-dimensional space and carefully deal
with the induced model misspecification. Our approach is conceptually very
different from existing works that can either only handle low-dimensional
linear bandits or passively deal with model misspecification. We showcase the
application of our approach to two pure exploration settings that were
previously under-studied: (1) the reward function belongs to a possibly
infinite-dimensional Reproducing Kernel Hilbert Space, and (2) the reward
function is nonlinear and can be approximated by neural networks. Our main
results provide sample complexity guarantees that only depend on the effective
dimension of the feature spaces in the kernel or neural representations.
Extensive experiments conducted on both synthetic and real-world datasets
demonstrate the efficacy of our methods.
