Understanding Notions of Stationarity in Non-Smooth Optimization
  Many contemporary applications in signal processing and machine learning give
rise to structured non-convex non-smooth optimization problems that can often
be tackled by simple iterative methods quite effectively. One of the keys to
understanding such a phenomenon---and, in fact, one of the very difficult
conundrums even for experts---lie in the study of "stationary points" of the
problem in question. Unlike smooth optimization, for which the definition of a
stationary point is rather standard, there is a myriad of definitions of
stationarity in non-smooth optimization. In this article, we give an
introduction to different stationarity concepts for several important classes
of non-convex non-smooth functions and discuss the geometric interpretations
and further clarify the relationship among these different concepts. We then
demonstrate the relevance of these constructions in some representative
applications and how they could affect the performance of iterative methods for
tackling these applications.
