Convergence Acceleration via Chebyshev Step: Plausible Interpretation of
  Deep-Unfolded Gradient Descent
  Deep unfolding is a promising deep-learning technique, whose network
architecture is based on expanding the recursive structure of existing
iterative algorithms. Although convergence acceleration is a remarkable
advantage of deep unfolding, its theoretical aspects have not been revealed
yet. The first half of this study details the theoretical analysis of the
convergence acceleration in deep-unfolded gradient descent (DUGD) whose
trainable parameters are step sizes. We propose a plausible interpretation of
the learned step-size parameters in DUGD by introducing the principle of
Chebyshev steps derived from Chebyshev polynomials. The use of Chebyshev steps
in gradient descent (GD) enables us to bound the spectral radius of a matrix
governing the convergence speed of GD, leading to a tight upper bound on the
convergence rate. The convergence rate of GD using Chebyshev steps is shown to
be asymptotically optimal, although it has no momentum terms. We also show that
Chebyshev steps numerically explain the learned step-size parameters in DUGD
well. In the second half of the study, %we apply the theory of Chebyshev steps
and Chebyshev-periodical successive over-relaxation (Chebyshev-PSOR) is
proposed for accelerating linear/nonlinear fixed-point iterations. Theoretical
analysis and numerical experiments indicate that Chebyshev-PSOR exhibits
significantly faster convergence for various examples such as Jacobi method and
proximal gradient methods.
