Local Clustering in Contextual Multi-Armed Bandits
  We study identifying user clusters in contextual multi-armed bandits (MAB).
Contextual MAB is an effective tool for many real applications, such as content
recommendation and online advertisement. In practice, user dependency plays an
essential role in the user's actions, and thus the rewards. Clustering similar
users can improve the quality of reward estimation, which in turn leads to more
effective content recommendation and targeted advertising. Different from
traditional clustering settings, we cluster users based on the unknown bandit
parameters, which will be estimated incrementally. In particular, we define the
problem of cluster detection in contextual MAB, and propose a bandit algorithm,
LOCB, embedded with local clustering procedure. And, we provide theoretical
analysis about LOCB in terms of the correctness and efficiency of clustering
and its regret bound. Finally, we evaluate the proposed algorithm from various
aspects, which outperforms state-of-the-art baselines.
