A Stochastic Subgradient Method for Distributionally Robust Non-Convex
  Learning
  We consider a distributionally robust formulation of stochastic optimization
problems arising in statistical learning, where robustness is with respect to
uncertainty in the underlying data distribution. Our formulation builds on
risk-averse optimization techniques and the theory of coherent risk measures.
It uses semi-deviation risk for quantifying uncertainty, allowing us to compute
solutions that are robust against perturbations in the population data
distribution. We consider a large family of loss functions that can be
non-convex and non-smooth and develop an efficient stochastic subgradient
method. We prove that it converges to a point satisfying the optimality
conditions. To our knowledge, this is the first method with rigorous
convergence guarantees in the context of non-convex non-smooth distributionally
robust stochastic optimization. Our method can achieve any desired level of
robustness with little extra computational cost compared to population risk
minimization. We also illustrate the performance of our algorithm on real
datasets arising in convex and non-convex supervised learning problems.
