The Symmetry between Arms and Knapsacks: A Primal-Dual Approach for
  Bandits with Knapsacks
  In this paper, we study the bandits with knapsacks (BwK) problem and develop
a primal-dual based algorithm that achieves a problem-dependent logarithmic
regret bound. The BwK problem extends the multi-arm bandit (MAB) problem to
model the resource consumption associated with playing each arm, and the
existing BwK literature has been mainly focused on deriving asymptotically
optimal distribution-free regret bounds. We first study the primal and dual
linear programs underlying the BwK problem. From this primal-dual perspective,
we discover symmetry between arms and knapsacks, and then propose a new notion
of sub-optimality measure for the BwK problem. The sub-optimality measure
highlights the important role of knapsacks in determining algorithm regret and
inspires the design of our two-phase algorithm. In the first phase, the
algorithm identifies the optimal arms and the binding knapsacks, and in the
second phase, it exhausts the binding knapsacks via playing the optimal arms
through an adaptive procedure. Our regret upper bound involves the proposed
sub-optimality measure and it has a logarithmic dependence on length of horizon
$T$ and a polynomial dependence on $m$ (the numbers of arms) and $d$ (the
number of knapsacks). To the best of our knowledge, this is the first
problem-dependent logarithmic regret bound for solving the general BwK problem.
