Efficient and Optimal Algorithms for Contextual Dueling Bandits under
  Realizability
  We study the $K$-armed contextual dueling bandit problem, a sequential
decision making setting in which the learner uses contextual information to
make two decisions, but only observes \emph{preference-based feedback}
suggesting that one decision was better than the other. We focus on the regret
minimization problem under realizability, where the feedback is generated by a
pairwise preference matrix that is well-specified by a given function class
$\mathcal F$. We provide a new algorithm that achieves the optimal regret rate
for a new notion of best response regret, which is a strictly stronger
performance measure than those considered in prior works. The algorithm is also
computationally efficient, running in polynomial time assuming access to an
online oracle for square loss regression over $\mathcal F$. This resolves an
open problem of Dud\'ik et al. [2015] on oracle efficient, regret-optimal
algorithms for contextual dueling bandits.
