Decentralized Multi-Armed Bandit Can Outperform Classic Upper Confidence
  Bound
  This paper studies a decentralized multi-armed bandit problem in a
multi-agent network. The problem is simultaneously solved by N agents assuming
they face a common set of M arms and share the same mean of each arm's reward.
Each agent can receive information only from its neighbors, where the neighbor
relations among the agents are described by a directed graph whose vertices
represent agents and whose directed edges depict neighbor relations. A fully
decentralized multi-armed bandit algorithm is proposed for each agent, which
twists the classic consensus algorithm and upper confidence bound (UCB)
algorithm. It is shown that the algorithm guarantees each agent to achieve a
better logarithmic asymptotic regret than the classic UCB provided the neighbor
graph is strongly connected. The regret can be further improved if the neighbor
graph is undirected.
