On the Asymptotic Linear Convergence Speed of Anderson Acceleration
  Applied to ADMM
  Empirical results show that Anderson acceleration (AA) can be a powerful
mechanism to improve the asymptotic linear convergence speed of the Alternating
Direction Method of Multipliers (ADMM) when ADMM by itself converges linearly.
However, theoretical results to quantify this improvement do not exist yet. In
this paper we explain and quantify this improvement in linear asymptotic
convergence speed for the special case of a stationary version of AA applied to
ADMM. We do so by considering the spectral properties of the Jacobians of ADMM
and the stationary version of AA evaluated at the fixed point, where the
coefficients of the stationary AA method are computed such that its asymptotic
linear convergence factor is optimal. The optimal linear convergence factors of
this stationary AA-ADMM method are computed analytically or by optimization,
based on previous work on optimal stationary AA acceleration. Using this
spectral picture and those analytical results, our approach provides new
insight into how and by how much the stationary AA method can improve the
asymptotic linear convergence factor of ADMM. Numerical results also indicate
that the optimal linear convergence factor of the stationary AA methods gives a
useful estimate for the asymptotic linear convergence speed of the
non-stationary AA method that is used in practice.
