Random directions stochastic approximation with deterministic
  perturbations
  We introduce deterministic perturbation schemes for the recently proposed
random directions stochastic approximation (RDSA) [17], and propose new
first-order and second-order algorithms. In the latter case, these are the
first second-order algorithms to incorporate deterministic perturbations. We
show that the gradient and/or Hessian estimates in the resulting algorithms
with deterministic perturbations are asymptotically unbiased, so that the
algorithms are provably convergent. Furthermore, we derive convergence rates to
establish the superiority of the first-order and second-order algorithms, for
the special case of a convex and quadratic optimization problem, respectively.
Numerical experiments are used to validate the theoretical results.
