Constrained regret minimization for multi-criterion multi-armed bandits
  We consider a stochastic multi-armed bandit setting and study the problem of
regret minimization over a given time horizon, subject to a risk constraint.
Each arm is associated with an unknown cost/loss distribution. The learning
agent is characterized by a risk-appetite that she is willing to tolerate,
which we model using a pre-specified upper bound on the Conditional Value at
Risk (CVaR). An optimal arm is one that minimizes the expected loss, among
those arms that satisfy the CVaR constraint. The agent is interested in
minimizing the number of pulls of suboptimal arms, including the ones that are
'too risky.' For this problem, we propose a Risk-Constrained Lower Confidence
Bound (RC-LCB) algorithm, that guarantees logarithmic regret, i.e., the average
number of plays of all non-optimal arms is at most logarithmic in the horizon.
The algorithm also outputs a boolean flag that correctly identifies with high
probability, whether the given instance was feasible/infeasible with respect to
the risk constraint. We prove lower bounds on the performance of any
risk-constrained regret minimization algorithm and establish a fundamental
trade-off between regret minimization and feasibility identification. The
proposed algorithm and analyses can be readily generalized to solve constrained
multi-criterion optimization problems in the bandits setting.
