Inexact Proximal Gradient Methods for Non-convex and Non-smooth
  Optimization
  In machine learning research, the proximal gradient methods are popular for
solving various optimization problems with non-smooth regularization. Inexact
proximal gradient methods are extremely important when exactly solving the
proximal operator is time-consuming, or the proximal operator does not have an
analytic solution. However, existing inexact proximal gradient methods only
consider convex problems. The knowledge of inexact proximal gradient methods in
the non-convex setting is very limited. % Moreover, for some machine learning
models, there is still no proposed solver for exactly solving the proximal
operator. To address this challenge, in this paper, we first propose three
inexact proximal gradient algorithms, including the basic version and
Nesterov's accelerated version. After that, we provide the theoretical analysis
to the basic and Nesterov's accelerated versions. The theoretical results show
that our inexact proximal gradient algorithms can have the same convergence
rates as the ones of exact proximal gradient algorithms in the non-convex
setting.
  Finally, we show the applications of our inexact proximal gradient algorithms
on three representative non-convex learning problems. All experimental results
confirm the superiority of our new inexact proximal gradient algorithms.
