Optimal Methods for Risk Averse Distributed Optimization
  This paper studies the communication complexity of risk averse optimization
over a network. The problem generalizes the well-studied risk-neutral
finite-sum distributed optimization problem and its importance stems from the
need to handle risk in an uncertain environment. For algorithms in the
literature, there exists a gap in communication complexities for solving
risk-averse and risk-neutral problems. We propose two distributed algorithms,
namely the distributed risk averse optimization (DRAO) method and the
distributed risk averse optimization with sliding (DRAO-S) method, to close the
gap. Specifically, the DRAO method achieves the optimal communication
complexity by assuming a certain saddle point subproblem can be easily solved
in the server node. The DRAO-S method removes the strong assumption by
introducing a novel saddle point sliding subroutine which only requires the
projection over the ambiguity set $P$. We observe that the number of
$P$-projections performed by DRAO-S is optimal. Moreover, we develop matching
lower complexity bounds to show that communication complexities of both DRAO
and DRAO-S are not improvable. Numerical experiments are conducted to
demonstrate the encouraging empirical performance of the DRAO-S method.
