Anderson acceleration of coordinate descent
  Acceleration of first order methods is mainly obtained via inertial
techniques \`a la Nesterov, or via nonlinear extrapolation. The latter has
known a recent surge of interest, with successful applications to gradient and
proximal gradient techniques. On multiple Machine Learning problems, coordinate
descent achieves performance significantly superior to full-gradient methods.
Speeding up coordinate descent in practice is not easy: inertially accelerated
versions of coordinate descent are theoretically accelerated, but might not
always lead to practical speed-ups. We propose an accelerated version of
coordinate descent using extrapolation, showing considerable speed up in
practice, compared to inertial accelerated coordinate descent and extrapolated
(proximal) gradient descent. Experiments on least squares, Lasso, elastic net
and logistic regression validate the approach.
