Sequential Subspace Search for Functional Bayesian Optimization
  Incorporating Experimenter Intuition
  We propose an algorithm for Bayesian functional optimisation - that is,
finding the function to optimise a process - guided by experimenter beliefs and
intuitions regarding the expected characteristics (length-scale, smoothness,
cyclicity etc.) of the optimal solution encoded into the covariance function of
a Gaussian Process. Our algorithm generates a sequence of finite-dimensional
random subspaces of functional space spanned by a set of draws from the
experimenter's Gaussian Process. Standard Bayesian optimisation is applied on
each subspace, and the best solution found used as a starting point (origin)
for the next subspace. Using the concept of effective dimensionality, we
analyse the convergence of our algorithm and provide a regret bound to show
that our algorithm converges in sub-linear time provided a finite effective
dimension exists. We test our algorithm in simulated and real-world
experiments, namely blind function matching, finding the optimal
precipitation-strengthening function for an aluminium alloy, and learning rate
schedule optimisation for deep networks.
