Optimizing Ranking Systems Online as Bandits
  Ranking system is the core part of modern retrieval and recommender systems,
where the goal is to rank candidate items given user contexts. Optimizing
ranking systems online means that the deployed system can serve user requests,
e.g., queries in the web search, and optimize the ranking policy by learning
from user interactions, e.g., clicks. Bandit is a general online learning
framework and can be used in our optimization task. However, due to the unique
features of ranking, there are several challenges in designing bandit
algorithms for ranking system optimization. In this dissertation, we study and
propose solutions for four challenges in optimizing ranking systems online:
effectiveness, safety, nonstationarity, and diversification. First, the
effectiveness is related to how fast the algorithm learns from interactions. We
study the effective online ranker evaluation task and propose the MergeDTS
algorithm to solve the problem effectively. Second, the deployed algorithm
should be safe, which means the algorithm only displays reasonable content to
user requests. To solve the safe online learning to rank problem, we propose
the BubbleRank algorithm. Third, as users change their preferences constantly,
the algorithm should handle the nonstationarity. We formulate this
nonstationary online learning to rank problem as cascade non-stationary bandits
and propose CascadeDUCB and CascadeSWUCB algorithms to solve the problem.
Finally, the contents in ranked lists should be diverse. We consider the
results diversification task and propose the CascadeHybird algorithm that
considers both the item relevance and results diversification when learning
from user interactions.
