Counterfactual Learning To Rank for Utility-Maximizing Query
  Autocompletion
  Conventional methods for query autocompletion aim to predict which completed
query a user will select from a list. A shortcoming of this approach is that
users often do not know which query will provide the best retrieval performance
on the current information retrieval system, meaning that any query
autocompletion methods trained to mimic user behavior can lead to suboptimal
query suggestions. To overcome this limitation, we propose a new approach that
explicitly optimizes the query suggestions for downstream retrieval
performance. We formulate this as a problem of ranking a set of rankings, where
each query suggestion is represented by the downstream item ranking it
produces. We then present a learning method that ranks query suggestions by the
quality of their item rankings. The algorithm is based on a counterfactual
learning approach that is able to leverage feedback on the items (e.g., clicks,
purchases) to evaluate query suggestions through an unbiased estimator, thus
avoiding the assumption that users write or select optimal queries. We
establish theoretical support for the proposed approach and provide
learning-theoretic guarantees. We also present empirical results on publicly
available datasets, and demonstrate real-world applicability using data from an
online shopping store.
