A comparison of mixed-variables Bayesian optimization approaches
  Most real optimization problems are defined over a mixed search space where
the variables are both discrete and continuous. In engineering applications,
the objective function is typically calculated with a numerically costly
black-box simulation.General mixed and costly optimization problems are
therefore of a great practical interest, yet their resolution remains in a
large part an open scientific question. In this article, costly mixed problems
are approached through Gaussian processes where the discrete variables are
relaxed into continuous latent variables. The continuous space is more easily
harvested by classical Bayesian optimization techniques than a mixed space
would. Discrete variables are recovered either subsequently to the continuous
optimization, or simultaneously with an additional continuous-discrete
compatibility constraint that is handled with augmented Lagrangians. Several
possible implementations of such Bayesian mixed optimizers are compared. In
particular, the reformulation of the problem with continuous latent variables
is put in competition with searches working directly in the mixed space. Among
the algorithms involving latent variables and an augmented Lagrangian, a
particular attention is devoted to the Lagrange multipliers for which a local
and a global estimation techniques are studied. The comparisons are based on
the repeated optimization of three analytical functions and a beam design
problem.
