Distributionally Robust Bayesian Optimization with $\phi$-divergences
  The study of robustness has received much attention due to its inevitability
in data-driven settings where many systems face uncertainty. One such example
of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted,
yet there only exists a limited number of works dedicated to this direction. In
particular, there is the work of Kirschner et al. (2020), which bridges the
existing literature of Distributionally Robust Optimization (DRO) by casting
the BO problem from the lens of DRO. While this work is pioneering, it
admittedly suffers from various practical shortcomings such as finite contexts
assumptions, leaving behind the main question Can one devise a computationally
tractable algorithm for solving this DRO-BO problem? In this work, we tackle
this question to a large degree of generality by considering robustness against
data-shift in $\phi$-divergences, which subsumes many popular choices, such as
the $\chi^2$-divergence, Total Variation, and the extant Kullback-Leibler (KL)
divergence. We show that the DRO-BO problem in this setting is equivalent to a
finite-dimensional optimization problem which, even in the continuous context
setting, can be easily implemented with provable sublinear regret bounds. We
then show experimentally that our method surpasses existing methods, attesting
to the theoretical results
