Linear Contextual Bandits with Knapsacks
  We consider the linear contextual bandit problem with resource consumption,
in addition to reward generation. In each round, the outcome of pulling an arm
is a reward as well as a vector of resource consumptions. The expected values
of these outcomes depend linearly on the context of that arm. The
budget/capacity constraints require that the total consumption doesn't exceed
the budget for each resource. The objective is once again to maximize the total
reward. This problem turns out to be a common generalization of classic linear
contextual bandits (linContextual), bandits with knapsacks (BwK), and the
online stochastic packing problem (OSPP). We present algorithms with
near-optimal regret bounds for this problem. Our bounds compare favorably to
results on the unstructured version of the problem where the relation between
the contexts and the outcomes could be arbitrary, but the algorithm only
competes against a fixed set of policies accessible through an optimization
oracle. We combine techniques from the work on linContextual, BwK, and OSPP in
a nontrivial manner while also tackling new difficulties that are not present
in any of these special cases.
