The Combinatorial Multi-Bandit Problem and its Application to Energy
  Management
  We study a Combinatorial Multi-Bandit Problem motivated by applications in
energy systems management. Given multiple probabilistic multi-arm bandits with
unknown outcome distributions, the task is to optimize the value of a
combinatorial objective function mapping the vector of individual bandit
outcomes to a single scalar reward. Unlike in single-bandit problems with
multi-dimensional action space, the outcomes of the individual bandits are
observable in our setting and the objective function is known. Guided by the
hypothesis that individual observability enables better trade-offs between
exploration and exploitation, we generalize the lower regret bound for single
bandits, showing that indeed for multiple bandits it admits parallelized
exploration. For our energy management application we propose a range of
algorithms that combine exploration principles for multi-arm bandits with
mathematical programming. In an experimental study we demonstrate the
effectiveness of our approach to learn action assignments for 150 bandits, each
having 24 actions, within a horizon of 365 episodes.
