Online convex optimization in the bandit setting: gradient descent
  without a gradient
  We consider a the general online convex optimization framework introduced by
Zinkevich. In this setting, there is a sequence of convex functions. Each
period, we must choose a signle point (from some feasible set) and pay a cost
equal to the value of the next function on our chosen point. Zinkevich shows
that, if the each function is revealed after the choice is made, then one can
achieve vanishingly small regret relative the best single decision chosen in
hindsight.
  We extend this to the bandit setting where we do not find out the entire
functions but rather just their value at our chosen point. We show how to get
vanishingly small regret in this setting.
  Our approach uses a simple approximation of the gradient that is computed
from evaluating a function at a single (random) point. We show that this
estimate is sufficient to mimic Zinkevich's gradient descent online analysis,
with access to the gradient (only being able to evaluate the function at a
single point).
