Convex-Concave Backtracking for Inertial Bregman Proximal Gradient
  Algorithms in Non-Convex Optimization
  Backtracking line-search is an old yet powerful strategy for finding a better
step sizes to be used in proximal gradient algorithms. The main principle is to
locally find a simple convex upper bound of the objective function, which in
turn controls the step size that is used. In case of inertial proximal gradient
algorithms, the situation becomes much more difficult and usually leads to very
restrictive rules on the extrapolation parameter. In this paper, we show that
the extrapolation parameter can be controlled by locally finding also a simple
concave lower bound of the objective function. This gives rise to a double
convex-concave backtracking procedure which allows for an adaptive choice of
both the step size and extrapolation parameters. We apply this procedure to the
class of inertial Bregman proximal gradient methods, and prove that any
sequence generated by these algorithms converges globally to a critical point
of the function at hand. Numerical experiments on a number of challenging
non-convex problems in image processing and machine learning were conducted and
show the power of combining inertial step and double backtracking strategy in
achieving improved performances.
