Forced-exploration free Strategies for Unimodal Bandits
  We consider a multi-armed bandit problem specified by a set of Gaussian or
Bernoulli distributions endowed with a unimodal structure. Although this
problem has been addressed in the literature (Combes and Proutiere, 2014), the
state-of-the-art algorithms for such structure make appear a forced-exploration
mechanism. We introduce IMED-UB, the first forced-exploration free strategy
that exploits the unimodal-structure, by adapting to this setting the Indexed
Minimum Empirical Divergence (IMED) strategy introduced by Honda and Takemura
(2015). This strategy is proven optimal. We then derive KLUCB-UB, a KLUCB
version of IMED-UB, which is also proven optimal. Owing to our proof technique,
we are further able to provide a concise finite-time analysis of both
strategies in an unified way. Numerical experiments show that both IMED-UB and
KLUCB-UB perform similarly in practice and outperform the state-of-the-art
algorithms.
