A globally convergent fast iterative shrinkage-thresholding algorithm with a new momentum factor for single and multi-objective convex optimization
Convex-composite optimization, which minimizes an objective function
represented by the sum of a differentiable function and a convex one, is widely
used in machine learning and signal/image processing. Fast Iterative Shrinkage
Thresholding Algorithm (FISTA) is a typical method for solving this problem and
has a global convergence rate of $O(1 / k^2)$. Recently, this has been extended
to multi-objective optimization, together with the proof of the $O(1 / k^2)$
global convergence rate. However, its momentum factor is classical, and the
convergence of its iterates has not been proven. In this work, introducing some
additional hyperparameters $(a, b)$, we propose another accelerated proximal
gradient method with a general momentum factor, which is new even for the
single-objective cases. We show that our proposed method also has a global
convergence rate of $O(1/k^2)$ for any $(a,b)$, and further that the generated
sequence of iterates converges to a weak Pareto solution when $a$ is positive,
an essential property for the finite-time manifold identification. Moreover, we
report numerical results with various $(a,b)$, showing that some of these
choices give better results than the classical momentum factors.