Bandits with Delayed, Aggregated Anonymous Feedback
  We study a variant of the stochastic $K$-armed bandit problem, which we call
"bandits with delayed, aggregated anonymous feedback". In this problem, when
the player pulls an arm, a reward is generated, however it is not immediately
observed. Instead, at the end of each round the player observes only the sum of
a number of previously generated rewards which happen to arrive in the given
round. The rewards are stochastically delayed and due to the aggregated nature
of the observations, the information of which arm led to a particular reward is
lost. The question is what is the cost of the information loss due to this
delayed, aggregated anonymous feedback? Previous works have studied bandits
with stochastic, non-anonymous delays and found that the regret increases only
by an additive factor relating to the expected delay. In this paper, we show
that this additive regret increase can be maintained in the harder delayed,
aggregated anonymous feedback setting when the expected delay (or a bound on
it) is known. We provide an algorithm that matches the worst case regret of the
non-anonymous problem exactly when the delays are bounded, and up to
logarithmic factors or an additive variance term for unbounded delays.
