Primal-Dual Sequential Subspace Optimization for Saddle-point Problems
  We introduce a new sequential subspace optimization method for large-scale
saddle-point problems. It solves iteratively a sequence of auxiliary
saddle-point problems in low-dimensional subspaces, spanned by directions
derived from first-order information over the primal \emph{and} dual variables.
Proximal regularization is further deployed to stabilize the optimization
process. Experimental results demonstrate significantly better convergence
relative to popular first-order methods. We analyze the influence of the
subspace on the convergence of the algorithm, and assess its performance in
various deterministic optimization scenarios, such as bi-linear games,
ADMM-based constrained optimization and generative adversarial networks.
