Generalized Thompson Sampling for Contextual Bandits
  Thompson Sampling, one of the oldest heuristics for solving multi-armed
bandits, has recently been shown to demonstrate state-of-the-art performance.
The empirical success has led to great interests in theoretical understanding
of this heuristic. In this paper, we approach this problem in a way very
different from existing efforts. In particular, motivated by the connection
between Thompson Sampling and exponentiated updates, we propose a new family of
algorithms called Generalized Thompson Sampling in the expert-learning
framework, which includes Thompson Sampling as a special case. Similar to most
expert-learning algorithms, Generalized Thompson Sampling uses a loss function
to adjust the experts' weights. General regret bounds are derived, which are
also instantiated to two important loss functions: square loss and logarithmic
loss. In contrast to existing bounds, our results apply to quite general
contextual bandits. More importantly, they quantify the effect of the "prior"
distribution on the regret bounds.
