A Stochastic Quasi-Newton Method with Nesterov's Accelerated Gradient
  Incorporating second order curvature information in gradient based methods
have shown to improve convergence drastically despite its computational
intensity. In this paper, we propose a stochastic (online) quasi-Newton method
with Nesterov's accelerated gradient in both its full and limited memory forms
for solving large scale non-convex optimization problems in neural networks.
The performance of the proposed algorithm is evaluated in Tensorflow on
benchmark classification and regression problems. The results show improved
performance compared to the classical second order oBFGS and oLBFGS methods and
popular first order stochastic methods such as SGD and Adam. The performance
with different momentum rates and batch sizes have also been illustrated.
