Directed particle swarm optimization with Gaussian-process-based
  function forecasting
  Particle swarm optimization (PSO) is an iterative search method that moves a
set of candidate solution around a search-space towards the best known global
and local solutions with randomized step lengths. PSO frequently accelerates
optimization in practical applications, where gradients are not available and
function evaluations expensive. Yet the traditional PSO algorithm ignores the
potential knowledge that could have been gained of the objective function from
the observations by individual particles. Hence, we draw upon concepts from
Bayesian optimization and introduce a stochastic surrogate model of the
objective function. That is, we fit a Gaussian process to past evaluations of
the objective function, forecast its shape and then adapt the particle
movements based on it. Our computational experiments demonstrate that baseline
implementations of PSO (i.e., SPSO2011) are outperformed. Furthermore, compared
to, state-of-art surrogate-assisted evolutionary algorithms, we achieve
substantial performance improvements on several popular benchmark functions.
Overall, we find that our algorithm attains desirable properties for
exploratory and exploitative behavior.
