Wasserstein Distributionally Robust Optimization and Variation
  Regularization
  Wasserstein distributionally robust optimization (DRO) has recently achieved
empirical success for various applications in operations research and machine
learning, owing partly to its regularization effect. Although connection
between Wasserstein DRO and regularization has been established in several
settings, existing results often require restrictive assumptions, such as
smoothness or convexity, that are not satisfied for many problems. In this
paper, we develop a general theory on the variation regularization effect of
the Wasserstein DRO - a new form of regularization that generalizes
total-variation regularization, Lipschitz regularization and gradient
regularization. Our results cover possibly non-convex and non-smooth losses and
losses on non-Euclidean spaces. Examples include multi-item newsvendor,
portfolio selection, linear prediction, neural networks, manifold learning, and
intensity estimation for Poisson processes, etc. As an application of our
theory of variation regularization, we derive new generalization guarantees for
adversarial robust learning.
