Low Regret Binary Sampling Method for Efficient Global Optimization of
  Univariate Functions
  In this work, we propose a computationally efficient algorithm for the
problem of global optimization in univariate loss functions. For the
performance evaluation, we study the cumulative regret of the algorithm instead
of the simple regret between our best query and the optimal value of the
objective function. Although our approach has similar regret results with the
traditional lower-bounding algorithms such as the Piyavskii-Shubert method for
the Lipschitz continuous or Lipschitz smooth functions, it has a major
computational cost advantage. In Piyavskii-Shubert method, for certain types of
functions, the query points may be hard to determine (as they are solutions to
additional optimization problems). However, this issue is circumvented in our
binary sampling approach, where the sampling set is predetermined irrespective
of the function characteristics. For a search space of $[0,1]$, our approach
has at most $L\log (3T)$ and $2.25H$ regret for $L$-Lipschitz continuous and
$H$-Lipschitz smooth functions respectively. We also analytically extend our
results for a broader class of functions that covers more complex regularity
conditions.
