Dueling Bandits with Qualitative Feedback
  We formulate and study a novel multi-armed bandit problem called the
qualitative dueling bandit (QDB) problem, where an agent observes not numeric
but qualitative feedback by pulling each arm. We employ the same regret as the
dueling bandit (DB) problem where the duel is carried out by comparing the
qualitative feedback. Although we can naively use classic DB algorithms for
solving the QDB problem, this reduction significantly worsens the
performance---actually, in the QDB problem, the probability that one arm wins
the duel over another arm can be directly estimated without carrying out actual
duels. In this paper, we propose such direct algorithms for the QDB problem.
Our theoretical analysis shows that the proposed algorithms significantly
outperform DB algorithms by incorporating the qualitative feedback, and
experimental results also demonstrate vast improvement over the existing DB
algorithms.
