Beyond the Click-Through Rate: Web Link Selection with Multi-level
  Feedback
  The web link selection problem is to select a small subset of web links from
a large web link pool, and to place the selected links on a web page that can
only accommodate a limited number of links, e.g., advertisements,
recommendations, or news feeds. Despite the long concerned click-through rate
which reflects the attractiveness of the link itself, the revenue can only be
obtained from user actions after clicks, e.g., purchasing after being directed
to the product pages by recommendation links. Thus, the web links have an
intrinsic \emph{multi-level feedback structure}. With this observation, we
consider the context-free web link selection problem, where the objective is to
maximize revenue while ensuring that the attractiveness is no less than a
preset threshold. The key challenge of the problem is that each link's
multi-level feedbacks are stochastic, and unobservable unless the link is
selected. We model this problem with a constrained stochastic multi-armed
bandit formulation, and design an efficient link selection algorithm, called
Constrained Upper Confidence Bound algorithm (\textbf{Con-UCB}), and prove
$O(\sqrt{T\ln T})$ bounds on both the regret and the violation of the
attractiveness constraint. We conduct extensive experiments on three real-world
datasets, and show that \textbf{Con-UCB} outperforms state-of-the-art
context-free bandit algorithms concerning the multi-level feedback structure.
