Latent Bandits Revisited
  A latent bandit problem is one in which the learning agent knows the arm
reward distributions conditioned on an unknown discrete latent state. The
primary goal of the agent is to identify the latent state, after which it can
act optimally. This setting is a natural midpoint between online and offline
learning---complex models can be learned offline with the agent identifying
latent state online---of practical relevance in, say, recommender systems. In
this work, we propose general algorithms for this setting, based on both upper
confidence bounds (UCBs) and Thompson sampling. Our methods are contextual and
aware of model uncertainty and misspecification. We provide a unified
theoretical analysis of our algorithms, which have lower regret than classic
bandit policies when the number of latent states is smaller than actions. A
comprehensive empirical study showcases the advantages of our approach.
