Why Non-myopic Bayesian Optimization is Promising and How Far Should We
  Look-ahead? A Study via Rollout
  Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find
optimal sampling policies through solving a dynamic programming (DP)
formulation that maximizes a long-term reward over a rolling horizon. Though
promising, lookahead BO faces the risk of error propagation through its
increased dependence on a possibly mis-specified model. In this work we focus
on the rollout approximation for solving the intractable DP. We first prove the
improving nature of rollout in tackling lookahead BO and provide a sufficient
condition for the used heuristic to be rollout improving. We then provide both
a theoretical and practical guideline to decide on the rolling horizon
stagewise. This guideline is built on quantifying the negative effect of a
mis-specified model. To illustrate our idea, we provide case studies on both
single and multi-information source BO. Empirical results show the advantageous
properties of our method over several myopic and non-myopic BO algorithms.
