Unifying the stochastic and the adversarial Bandits with Knapsack
  This paper investigates the adversarial Bandits with Knapsack (BwK) online
learning problem, where a player repeatedly chooses to perform an action, pays
the corresponding cost, and receives a reward associated with the action. The
player is constrained by the maximum budget $B$ that can be spent to perform
actions, and the rewards and the costs of the actions are assigned by an
adversary. This problem has only been studied in the restricted setting where
the reward of an action is greater than the cost of the action, while we
provide a solution in the general setting. Namely, we propose EXP3.BwK, a novel
algorithm that achieves order optimal regret. We also propose EXP3++.BwK, which
is order optimal in the adversarial BwK setup, and incurs an almost optimal
expected regret with an additional factor of $\log(B)$ in the stochastic BwK
setup. Finally, we investigate the case of having large costs for the actions
(i.e., they are comparable to the budget size $B$), and show that for the
adversarial setting, achievable regret bounds can be significantly worse,
compared to the case of having costs bounded by a constant, which is a common
assumption within the BwK literature.
