Weak error analysis for stochastic gradient descent optimization
  algorithms
  Stochastic gradient descent (SGD) type optimization schemes are fundamental
ingredients in a large number of machine learning based algorithms. In
particular, SGD type optimization schemes are frequently employed in
applications involving natural language processing, object and face
recognition, fraud detection, computational advertisement, and numerical
approximations of partial differential equations. In mathematical convergence
results for SGD type optimization schemes there are usually two types of error
criteria studied in the scientific literature, that is, the error in the strong
sense and the error with respect to the objective function. In applications one
is often not only interested in the size of the error with respect to the
objective function but also in the size of the error with respect to a test
function which is possibly different from the objective function. The analysis
of the size of this error is the subject of this article. In particular, the
main result of this article proves under suitable assumptions that the size of
this error decays at the same speed as in the special case where the test
function coincides with the objective function.
