MIMICS: A Large-Scale Data Collection for Search Clarification
  Search clarification has recently attracted much attention due to its
applications in search engines. It has also been recognized as a major
component in conversational information seeking systems. Despite its
importance, the research community still feels the lack of a large-scale data
for studying different aspects of search clarification. In this paper, we
introduce MIMICS, a collection of search clarification datasets for real web
search queries sampled from the Bing query logs. Each clarification in MIMICS
is generated by a Bing production algorithm and consists of a clarifying
question and up to five candidate answers. MIMICS contains three datasets: (1)
MIMICS-Click includes over 400k unique queries, their associated clarification
panes, and the corresponding aggregated user interaction signals (i.e.,
clicks). (2) MIMICS-ClickExplore is an exploration data that includes
aggregated user interaction signals for over 60k unique queries, each with
multiple clarification panes. (3) MIMICS-Manual includes over 2k unique real
search queries. Each query-clarification pair in this dataset has been manually
labeled by at least three trained annotators. It contains graded quality labels
for the clarifying question, the candidate answer set, and the landing result
page for each candidate answer.
  MIMICS is publicly available for research purposes, thus enables researchers
to study a number of tasks related to search clarification, including
clarification generation and selection, user engagement prediction for
clarification, click models for clarification, and analyzing user interactions
with search clarification.
