Stochastic Contextual Dueling Bandits under Linear Stochastic
  Transitivity Models
  We consider the regret minimization task in a dueling bandits problem with
context information. In every round of the sequential decision problem, the
learner makes a context-dependent selection of two choice alternatives (arms)
to be compared with each other and receives feedback in the form of noisy
preference information. We assume that the feedback process is determined by a
linear stochastic transitivity model with contextualized utilities (CoLST), and
the learner's task is to include the best arm (with highest latent
context-dependent utility) in the duel. We propose a computationally efficient
algorithm, $\texttt{CoLSTIM}$, which makes its choice based on imitating the
feedback process using perturbed context-dependent utility estimates of the
underlying CoLST model. If each arm is associated with a $d$-dimensional
feature vector, we show that $\texttt{CoLSTIM}$ achieves a regret of order
$\tilde O( \sqrt{dT})$ after $T$ learning rounds. Additionally, we also
establish the optimality of $\texttt{CoLSTIM}$ by showing a lower bound for the
weak regret that refines the existing average regret analysis. Our experiments
demonstrate its superiority over state-of-art algorithms for special cases of
CoLST models.
