Robustness of accelerated first-order algorithms for strongly convex
  optimization problems
  We study the robustness of accelerated first-order algorithms to stochastic
uncertainties in gradient evaluation. Specifically, for unconstrained, smooth,
strongly convex optimization problems, we examine the mean-squared error in the
optimization variable when the iterates are perturbed by additive white noise.
This type of uncertainty may arise in situations where an approximation of the
gradient is sought through measurements of a real system or in a distributed
computation over a network. Even though the underlying dynamics of first-order
algorithms for this class of problems are nonlinear, we establish upper bounds
on the mean-squared deviation from the optimal solution that are tight up to
constant factors. Our analysis quantifies fundamental trade-offs between noise
amplification and convergence rates obtained via any acceleration scheme
similar to Nesterov's or heavy-ball methods. To gain additional analytical
insight, for strongly convex quadratic problems, we explicitly evaluate the
steady-state variance of the optimization variable in terms of the eigenvalues
of the Hessian of the objective function. We demonstrate that the entire
spectrum of the Hessian, rather than just the extreme eigenvalues, influence
robustness of noisy algorithms. We specialize this result to the problem of
distributed averaging over undirected networks and examine the role of network
size and topology on the robustness of noisy accelerated algorithms.
