Regret and Belief Complexity Trade-off in Gaussian Process Bandits via
  Information Thresholding
  Bayesian optimization is a framework for global search via maximum a
posteriori updates rather than simulated annealing, and has gained prominence
for decision-making under uncertainty. In this work, we cast Bayesian
optimization as a multi-armed bandit problem, where the payoff function is
sampled from a Gaussian process (GP). Further, we focus on action selections
via upper confidence bound (UCB) or expected improvement (EI) due to their
prevalent use in practice. Prior works using GPs for bandits cannot allow the
iteration horizon $T$ to be large, as the complexity of computing the posterior
parameters scales cubically with the number of past observations. To circumvent
this computational burden, we propose a simple statistical test: only
incorporate an action into the GP posterior when its conditional entropy
exceeds an $\epsilon$ threshold. Doing so permits us to precisely characterize
the trade-off between regret bounds of GP bandit algorithms and complexity of
the posterior distributions depending on the compression parameter $\epsilon$
for both discrete and continuous action sets. To best of our knowledge, this is
the first result which allows us to obtain sublinear regret bounds while still
maintaining sublinear growth rate of the complexity of the posterior which is
linear in the existing literature. Moreover, a provably finite bound on the
complexity could be achieved but the algorithm would result in
$\epsilon$-regret which means $\textbf{Reg}_T/T \rightarrow
\mathcal{O}(\epsilon)$ as $T\rightarrow \infty$. Experimentally, we observe
state of the art accuracy and complexity trade-offs for GP bandit algorithms
applied to global optimization, suggesting the merits of compressed GPs in
bandit settings.
