Optimal Algorithms for Stochastic Multi-Armed Bandits with Heavy Tailed
  Rewards
  In this paper, we consider stochastic multi-armed bandits (MABs) with
heavy-tailed rewards, whose $p$-th moment is bounded by a constant $\nu_{p}$
for $1<p\leq2$. First, we propose a novel robust estimator which does not
require $\nu_{p}$ as prior information, while other existing robust estimators
demand prior knowledge about $\nu_{p}$. We show that an error probability of
the proposed estimator decays exponentially fast. Using this estimator, we
propose a perturbation-based exploration strategy and develop a generalized
regret analysis scheme that provides upper and lower regret bounds by revealing
the relationship between the regret and the cumulative density function of the
perturbation. From the proposed analysis scheme, we obtain gap-dependent and
gap-independent upper and lower regret bounds of various perturbations. We also
find the optimal hyperparameters for each perturbation, which can achieve the
minimax optimal regret bound with respect to total rounds. In simulation, the
proposed estimator shows favorable performance compared to existing robust
estimators for various $p$ values and, for MAB problems, the proposed
perturbation strategy outperforms existing exploration methods.
