Convergence of stochastic gradient descent schemes for
  Lojasiewicz-landscapes
  In this article, we consider convergence of stochastic gradient descent
schemes (SGD) under weak assumptions on the underlying landscape. More
explicitly, we show that on the event that the SGD stays local we have
convergence of the SGD if there is only a countable number of critical points
or if the target function/landscape satisfies Lojasiewicz-inequalities around
all critical levels as all analytic functions do. In particular, we show that
for neural networks with analytic activation function such as softplus, sigmoid
and the hyperbolic tangent, SGD converges on the event of staying local, if the
random variables modeling the signal and response in the training are compactly
supported.
