A Tutorial on Thompson Sampling
  Thompson sampling is an algorithm for online decision problems where actions
are taken sequentially in a manner that must balance between exploiting what is
known to maximize immediate performance and investing to accumulate new
information that may improve future performance. The algorithm addresses a
broad range of problems in a computationally efficient manner and is therefore
enjoying wide use. This tutorial covers the algorithm and its application,
illustrating concepts through a range of examples, including Bernoulli bandit
problems, shortest path problems, product recommendation, assortment, active
learning with neural networks, and reinforcement learning in Markov decision
processes. Most of these problems involve complex information structures, where
information revealed by taking an action informs beliefs about other actions.
We will also discuss when and why Thompson sampling is or is not effective and
relations to alternative algorithms.
