A Comprehensive Study on Optimization Strategies for Gradient Descent In
  Deep Learning
  One of the most important parts of Artificial Neural Networks is minimizing
the loss functions which tells us how good or bad our model is. To minimize
these losses we need to tune the weights and biases. Also to calculate the
minimum value of a function we need gradient. And to update our weights we need
gradient descent. But there are some problems with regular gradient descent ie.
it is quite slow and not that accurate. This article aims to give an
introduction to optimization strategies to gradient descent. In addition, we
shall also discuss the architecture of these algorithms and further
optimization of Neural Networks in general
