Power Gradient Descent
  The development of machine learning is promoting the search for fast and
stable minimization algorithms. To this end, we suggest a change in the current
gradient descent methods that should speed up the motion in flat regions and
slow it down in steep directions of the function to minimize. It is based on a
"power gradient", in which each component of the gradient is replaced by its
versus-preserving $H$-th power, with $0<H<1$. We test three modern gradient
descent methods fed by such variant and by standard gradients, finding the new
version to achieve significantly better performances for the Nesterov
accelerated gradient and AMSGrad. We also propose an effective new take on the
ADAM algorithm, which includes power gradients with varying $H$.
