An Empirical Process Approach to the Union Bound: Practical Algorithms
  for Combinatorial and Linear Bandits
  This paper proposes near-optimal algorithms for the pure-exploration linear
bandit problem in the fixed confidence and fixed budget settings. Leveraging
ideas from the theory of suprema of empirical processes, we provide an
algorithm whose sample complexity scales with the geometry of the instance and
avoids an explicit union bound over the number of arms. Unlike previous
approaches which sample based on minimizing a worst-case variance (e.g.
G-optimal design), we define an experimental design objective based on the
Gaussian-width of the underlying arm set. We provide a novel lower bound in
terms of this objective that highlights its fundamental role in the sample
complexity. The sample complexity of our fixed confidence algorithm matches
this lower bound, and in addition is computationally efficient for
combinatorial classes, e.g. shortest-path, matchings and matroids, where the
arm sets can be exponentially large in the dimension. Finally, we propose the
first algorithm for linear bandits in the the fixed budget setting. Its
guarantee matches our lower bound up to logarithmic factors.
