A theoretical and empirical study of new adaptive algorithms with
  additional momentum steps and shifted updates for stochastic non-convex
  optimization
  In the following paper we introduce new adaptive algorithms endowed with
momentum terms for stochastic non-convex optimization problems. We investigate
the almost sure convergence to stationary points, along with a finite-time
horizon analysis with respect to a chosen final iteration, and we also inspect
the worst-case iteration complexity. An estimate for the expectation of the
squared Euclidean norm of the gradient is given and the theoretical analysis
that we perform is assisted by various computational simulations for neural
network training.
