SACOBRA with Online Whitening for Solving Optimization Problems with
  High Conditioning
  Real-world optimization problems often have expensive objective functions in
terms of cost and time. It is desirable to find near-optimal solutions with
very few function evaluations. Surrogate-assisted optimizers tend to reduce the
required number of function evaluations by replacing the real function with an
efficient mathematical model built on few evaluated points. Problems with a
high condition number are a challenge for many surrogate-assisted optimizers
including SACOBRA. To address such problems we propose a new online whitening
operating in the black-box optimization paradigm. We show on a set of
high-conditioning functions that online whitening tackles SACOBRA's early
stagnation issue and reduces the optimization error by a factor between 10 to
1e12 as compared to the plain SACOBRA, though it imposes many extra function
evaluations. Covariance matrix adaptation evolution strategy (CMA-ES) has for
very high numbers of function evaluations even lower errors, whereas SACOBRA
performs better in the expensive setting (less than 1e03 function evaluations).
If we count all parallelizable function evaluations (population evaluation in
CMA-ES, online whitening in our approach) as one iteration, then both
algorithms have comparable strength even on the long run. This holds for
problems with dimension D <= 20.
