Combined Global and Local Search for Optimization with Gaussian Process
  Models
  Gaussian process (GP) model based optimization is widely applied in
simulation and machine learning. In general, it first estimates a GP model
based on a few observations from the true response and then employs this model
to guide the search, aiming to quickly locate the global optimum. Despite its
successful applications, it has several limitations that may hinder its broader
usage. First, building an accurate GP model can be difficult and
computationally expensive, especially when the response function is multi-modal
or varies significantly over the design space. Second, even with an appropriate
model, the search process can be trapped in suboptimal regions before moving to
the global optimum due to the excessive effort spent around the current best
solution. In this work, we adopt the Additive Global and Local GP (AGLGP) model
in the optimization framework. The model is rooted in the inducing-points-based
GP sparse approximations and is combined with independent local models in
different regions. With these properties, the AGLGP model is suitable for
multi-modal responses with relatively large data sizes. Based on this AGLGP
model, we propose a Combined Global and Local search for Optimization (CGLO)
algorithm. It first divides the whole design space into disjoint local regions
and identifies a promising region with the global model. Next, a local model in
the selected region is fit to guide detailed search within this region. The
algorithm then switches back to the global step when a good local solution is
found. The global and local natures of CGLO enable it to enjoy the benefits of
both global and local search to efficiently locate the global optimum.
