Convergence Rates for Multi-classs Logistic Regression Near Minimum
  In the current paper we provide constructive estimation of the convergence
rate for training a known class of neural networks: the multi-class logistic
regression. Despite several decades of successful use, our rigorous results
appear new, reflective of the gap between practice and theory of machine
learning. Training a neural network is typically done via variations of the
gradient descent method. If a minimum of the loss function exists and gradient
descent is used as the training method, we provide an expression that relates
learning rate to the rate of convergence to the minimum. The method involves an
estimate of the condition number of the Hessian of the loss function. We also
discuss the existence of a minimum, as it is not automatic that a minimum
exists. One method of ensuring convergence is by assigning positive probabiity
to every class in the training dataset.
