Curvature-Aware Derivative-Free Optimization
  We propose a new line-search method, coined Curvature-Aware Random Search
(CARS), for derivative-free optimization. CARS exploits approximate curvature
information to estimate the optimal step-size given a search direction. We
prove that for strongly convex objective functions, CARS converges linearly if
the search direction is drawn from a distribution satisfying very mild
conditions. We also explore a variant, CARS-NQ, which uses Numerical Quadrature
instead of a Monte Carlo method when approximating curvature along the search
direction. We show CARS-NQ is effective on highly non-convex problems of the
form $f = f_{\mathrm{cvx}} + f_{\mathrm{osc}}$ where $f_{\mathrm{cvx}}$ is
strongly convex and $f_{\mathrm{osc}}$ is rapidly oscillating. Experimental
results show that CARS and CARS-NQ match or exceed the state-of-the-arts on
benchmark problem sets.
