Risk-Aware Algorithms for Adversarial Contextual Bandits
  In this work we consider adversarial contextual bandits with risk
constraints. At each round, nature prepares a context, a cost for each arm, and
additionally a risk for each arm. The learner leverages the context to pull an
arm and then receives the corresponding cost and risk associated with the
pulled arm. In addition to minimizing the cumulative cost, the learner also
needs to satisfy long-term risk constraints -- the average of the cumulative
risk from all pulled arms should not be larger than a pre-defined threshold. To
address this problem, we first study the full information setting where in each
round the learner receives an adversarial convex loss and a convex constraint.
We develop a meta algorithm leveraging online mirror descent for the full
information setting and extend it to contextual bandit with risk constraints
setting using expert advice. Our algorithms can achieve near-optimal regret in
terms of minimizing the total cost, while successfully maintaining a sublinear
growth of cumulative risk constraint violation.
