Stochastically Controlled Stochastic Gradient for the Convex and
  Non-convex Composition problem
  In this paper, we consider the convex and non-convex composition problem with
the structure $\frac{1}{n}\sum\nolimits_{i = 1}^n {{F_i}( {G( x )} )}$, where
$G( x )=\frac{1}{n}\sum\nolimits_{j = 1}^n {{G_j}( x )} $ is the inner
function, and $F_i(\cdot)$ is the outer function. We explore the variance
reduction based method to solve the composition optimization. Due to the fact
that when the number of inner function and outer function are large, it is not
reasonable to estimate them directly, thus we apply the stochastically
controlled stochastic gradient (SCSG) method to estimate the gradient of the
composition function and the value of the inner function. The query complexity
of our proposed method for the convex and non-convex problem is equal to or
better than the current method for the composition problem. Furthermore, we
also present the mini-batch version of the proposed method, which has the
improved the query complexity with related to the size of the mini-batch.
