Convergence of adaptive algorithms for weakly convex constrained
  optimization
  We analyze the adaptive first order algorithm AMSGrad, for solving a
constrained stochastic optimization problem with a weakly convex objective. We
prove the $\mathcal{\tilde O}(t^{-1/4})$ rate of convergence for the norm of
the gradient of Moreau envelope, which is the standard stationarity measure for
this class of problems. It matches the known rates that adaptive algorithms
enjoy for the specific case of unconstrained smooth stochastic optimization.
Our analysis works with mini-batch size of $1$, constant first and second order
moment parameters, and possibly unbounded optimization domains. Finally, we
illustrate the applications and extensions of our results to specific problems
and algorithms.
