Asynchronous Delay-Aware Accelerated Proximal Coordinate Descent for
  Nonconvex Nonsmooth Problems
  Nonconvex and nonsmooth problems have recently attracted considerable
attention in machine learning. However, developing efficient methods for the
nonconvex and nonsmooth optimization problems with certain performance
guarantee remains a challenge. Proximal coordinate descent (PCD) has been
widely used for solving optimization problems, but the knowledge of PCD methods
in the nonconvex setting is very limited. On the other hand, the asynchronous
proximal coordinate descent (APCD) recently have received much attention in
order to solve large-scale problems. However, the accelerated variants of APCD
algorithms are rarely studied. In this paper, we extend APCD method to the
accelerated algorithm (AAPCD) for nonsmooth and nonconvex problems that
satisfies the sufficient descent property, by comparing between the function
values at proximal update and a linear extrapolated point using a delay-aware
momentum value. To the best of our knowledge, we are the first to provide
stochastic and deterministic accelerated extension of APCD algorithms for
general nonconvex and nonsmooth problems ensuring that for both bounded delays
and unbounded delays every limit point is a critical point. By leveraging
Kurdyka-Lojasiewicz property, we will show linear and sublinear convergence
rates for the deterministic AAPCD with bounded delays. Numerical results
demonstrate the practical efficiency of our algorithm in speed.
