A Survey of Risk-Aware Multi-Armed Bandits
In several applications such as clinical trials and financial portfolio
optimization, the expected value (or the average reward) does not
satisfactorily capture the merits of a drug or a portfolio. In such
applications, risk plays a crucial role, and a risk-aware performance measure
is preferable, so as to capture losses in the case of adverse events. This
survey aims to consolidate and summarise the existing research on risk
measures, specifically in the context of multi-armed bandits. We review various
risk measures of interest, and comment on their properties. Next, we review
existing concentration inequalities for various risk measures. Then, we proceed
to defining risk-aware bandit problems, We consider algorithms for the regret
minimization setting, where the exploration-exploitation trade-off manifests,
as well as the best-arm identification setting, which is a pure exploration
problem -- both in the context of risk-sensitive measures. We conclude by
commenting on persisting challenges and fertile areas for future research.