An Algebraically Converging Stochastic Gradient Descent Algorithm for
  Global Optimization
  We propose a new stochastic gradient descent algorithm for finding the global
optimizer of nonconvex optimization problems, referred to here as "AdaVar". A
key component in the algorithm is the adaptive tuning of the randomness based
on the value of the objective function. In the language of simulated annealing,
the temperature is state-dependent. With this, we can prove global convergence
with an algebraic rate both in probability and in the parameter space. This is
a major improvement over the classical rate from using a simpler control of the
noise term. The convergence proof is based on the actual discrete setup of the
algorithm. We also present several numerical examples demonstrating the
efficiency and robustness of the algorithm for global convergence.
