Fourier Representations for Black-Box Optimization over Categorical
  Variables
  Optimization of real-world black-box functions defined over purely
categorical variables is an active area of research. In particular,
optimization and design of biological sequences with specific functional or
structural properties have a profound impact in medicine, materials science,
and biotechnology. Standalone search algorithms, such as simulated annealing
(SA) and Monte Carlo tree search (MCTS), are typically used for such
optimization problems. In order to improve the performance and sample
efficiency of such algorithms, we propose to use existing methods in
conjunction with a surrogate model for the black-box evaluations over purely
categorical variables. To this end, we present two different representations, a
group-theoretic Fourier expansion and an abridged one-hot encoded Boolean
Fourier expansion. To learn such representations, we consider two different
settings to update our surrogate model. First, we utilize an adversarial online
regression setting where Fourier characters of each representation are
considered as experts and their respective coefficients are updated via an
exponential weight update rule each time the black box is evaluated. Second, we
consider a Bayesian setting where queries are selected via Thompson sampling
and the posterior is updated via a sparse Bayesian regression model (over our
proposed representation) with a regularized horseshoe prior. Numerical
experiments over synthetic benchmarks as well as real-world RNA sequence
optimization and design problems demonstrate the representational power of the
proposed methods, which achieve competitive or superior performance compared to
state-of-the-art counterparts, while improving the computation cost and/or
sample efficiency, substantially.
