Correlated bandits or: How to minimize mean-squared error online
  While the objective in traditional multi-armed bandit problems is to find the
arm with the highest mean, in many settings, finding an arm that best captures
information about other arms is of interest. This objective, however, requires
learning the underlying correlation structure and not just the means of the
arms. Sensors placement for industrial surveillance and cellular network
monitoring are a few applications, where the underlying correlation structure
plays an important role. Motivated by such applications, we formulate the
correlated bandit problem, where the objective is to find the arm with the
lowest mean-squared error (MSE) in estimating all the arms. To this end, we
derive first an MSE estimator, based on sample variances and covariances, and
show that our estimator exponentially concentrates around the true MSE. Under a
best-arm identification framework, we propose a successive rejects type
algorithm and provide bounds on the probability of error in identifying the
best arm. Using minmax theory, we also derive fundamental performance limits
for the correlated bandit problem.
