High Quality Related Search Query Suggestions using Deep Reinforcement
  Learning
  "High Quality Related Search Query Suggestions" task aims at recommending
search queries which are real, accurate, diverse, relevant and engaging.
Obtaining large amounts of query-quality human annotations is expensive. Prior
work on supervised query suggestion models suffered from selection and exposure
bias, and relied on sparse and noisy immediate user-feedback (e.g., clicks),
leading to low quality suggestions. Reinforcement Learning techniques employed
to reformulate a query using terms from search results, have limited
scalability to large-scale industry applications. To recommend high quality
related search queries, we train a Deep Reinforcement Learning model to predict
the query a user would enter next. The reward signal is composed of long-term
session-based user feedback, syntactic relatedness and estimated naturalness of
generated query. Over the baseline supervised model, our proposed approach
achieves a significant relative improvement in terms of recommendation
diversity (3%), down-stream user-engagement (4.2%) and per-sentence word
repetitions (82%).
