Greedy Algorithm almost Dominates in Smoothed Contextual Bandits
  Online learning algorithms, widely used to power search and content
optimization on the web, must balance exploration and exploitation, potentially
sacrificing the experience of current users in order to gain information that
will lead to better decisions in the future. While necessary in the worst case,
explicit exploration has a number of disadvantages compared to the greedy
algorithm that always "exploits" by choosing an action that currently looks
optimal. We ask under what conditions inherent diversity in the data makes
explicit exploration unnecessary. We build on a recent line of work on the
smoothed analysis of the greedy algorithm in the linear contextual bandits
model. We improve on prior results to show that a greedy approach almost
matches the best possible Bayesian regret rate of any other algorithm on the
same problem instance whenever the diversity conditions hold, and that this
regret is at most $\tilde O(T^{1/3})$.
