Re-Examining Linear Embeddings for High-Dimensional Bayesian
  Optimization
  Bayesian optimization (BO) is a popular approach to optimize
expensive-to-evaluate black-box functions. A significant challenge in BO is to
scale to high-dimensional parameter spaces while retaining sample efficiency. A
solution considered in existing literature is to embed the high-dimensional
space in a lower-dimensional manifold, often via a random linear embedding. In
this paper, we identify several crucial issues and misconceptions about the use
of linear embeddings for BO. We study the properties of linear embeddings from
the literature and show that some of the design choices in current approaches
adversely impact their performance. We show empirically that properly
addressing these issues significantly improves the efficacy of linear
embeddings for BO on a range of problems, including learning a gait policy for
robot locomotion.
