Unbiased Learning to Rank: Online or Offline?
  How to obtain an unbiased ranking model by learning to rank with biased user
feedback is an important research question for IR. Existing work on unbiased
learning to rank (ULTR) can be broadly categorized into two groups -- the
studies on unbiased learning algorithms with logged data, namely the
\textit{offline} unbiased learning, and the studies on unbiased parameters
estimation with real-time user interactions, namely the \textit{online}
learning to rank. While their definitions of \textit{unbiasness} are different,
these two types of ULTR algorithms share the same goal -- to find the best
models that rank documents based on their intrinsic relevance or utility.
However, most studies on offline and online unbiased learning to rank are
carried in parallel without detailed comparisons on their background theories
and empirical performance. In this paper, we formalize the task of unbiased
learning to rank and show that existing algorithms for offline unbiased
learning and online learning to rank are just the two sides of the same coin.
We evaluate six state-of-the-art ULTR algorithms and find that most of them can
be used in both offline settings and online environments with or without minor
modifications. Further, we analyze how different offline and online learning
paradigms would affect the theoretical foundation and empirical effectiveness
of each algorithm on both synthetic and real search data. Our findings could
provide important insights and guideline for choosing and deploying ULTR
algorithms in practice.
