First-Order Methods for Convex Optimization
  First-order methods for solving convex optimization problems have been at the
forefront of mathematical optimization in the last 20 years. The rapid
development of this important class of algorithms is motivated by the success
stories reported in various applications, including most importantly machine
learning, signal processing, imaging and control theory. First-order methods
have the potential to provide low accuracy solutions at low computational
complexity which makes them an attractive set of tools in large-scale
optimization problems. In this survey we cover a number of key developments in
gradient-based optimization methods. This includes non-Euclidean extensions of
the classical proximal gradient method, and its accelerated versions.
Additionally we survey recent developments within the class of projection-free
methods, and proximal versions of primal-dual schemes. We give complete proofs
for various key results, and highlight the unifying aspects of several
optimization algorithms.
