Distributionally Robust Optimization and Generalization in Kernel
  Methods
  Distributionally robust optimization (DRO) has attracted attention in machine
learning due to its connections to regularization, generalization, and
robustness. Existing work has considered uncertainty sets based on
phi-divergences and Wasserstein distances, each of which have drawbacks. In
this paper, we study DRO with uncertainty sets measured via maximum mean
discrepancy (MMD). We show that MMD DRO is roughly equivalent to regularization
by the Hilbert norm and, as a byproduct, reveal deep connections to classic
results in statistical learning. In particular, we obtain an alternative proof
of a generalization bound for Gaussian kernel ridge regression via a DRO lense.
The proof also suggests a new regularizer. Our results apply beyond kernel
methods: we derive a generically applicable approximation of MMD DRO, and show
that it generalizes recent work on variance-based regularization.
