Contextual Bandits with Continuous Actions: Smoothing, Zooming, and
  Adapting
  We study contextual bandit learning with an abstract policy class and
continuous action space. We obtain two qualitatively different regret bounds:
one competes with a smoothed version of the policy class under no continuity
assumptions, while the other requires standard Lipschitz assumptions. Both
bounds exhibit data-dependent "zooming" behavior and, with no tuning, yield
improved guarantees for benign problems. We also study adapting to unknown
smoothness parameters, establishing a price-of-adaptivity and deriving optimal
adaptive algorithms that require no additional information.
