Stochastic Trust Region Inexact Newton Method for Large-scale Machine
  Learning
  Nowadays stochastic approximation methods are one of the major research
direction to deal with the large-scale machine learning problems. From
stochastic first order methods, now the focus is shifting to stochastic second
order methods due to their faster convergence and availability of computing
resources. In this paper, we have proposed a novel Stochastic Trust RegiOn
Inexact Newton method, called as STRON, to solve large-scale learning problems
which uses conjugate gradient (CG) to inexactly solve trust region subproblem.
The method uses progressive subsampling in the calculation of gradient and
Hessian values to take the advantage of both, stochastic and full-batch
regimes. We have extended STRON using existing variance reduction techniques to
deal with the noisy gradients and using preconditioned conjugate gradient (PCG)
as subproblem solver, and empirically proved that they do not work as expected,
for the large-scale learning problems. Finally, our empirical results prove
efficacy of the proposed method against existing methods with bench marked
datasets.
